Commit graph

17 commits

Author SHA1 Message Date
Ishaan Jaff
b2b3e40d13
(feat) use @google-cloud/vertexai js sdk with litellm (#6873)
* stash gemini JS test

* add vertex js sdj example

* handle vertex pass through separately

* tes vertex JS sdk

* fix vertex_proxy_route

* use PassThroughStreamingHandler

* fix PassThroughStreamingHandler

* use common _create_vertex_response_logging_payload_for_generate_content

* test vertex js

* add working vertex jest tests

* move basic bass through test

* use good name for test

* test vertex

* test_chunk_processor_yields_raw_bytes

* unit tests for streaming

* test_convert_raw_bytes_to_str_lines

* run unit tests 1st

* simplify local

* docs add usage example for js

* use get_litellm_virtual_key

* add unit tests for vertex pass through
2024-11-22 16:50:10 -08:00
Ishaan Jaff
b8af46e1a2
(feat) Add usage tracking for streaming /anthropic passthrough routes (#6842)
* use 1 file for AnthropicPassthroughLoggingHandler

* add support for anthropic streaming usage tracking

* ci/cd run again

* fix - add real streaming for anthropic pass through

* remove unused function stream_response

* working anthropic streaming logging

* fix code quality

* fix use 1 file for vertex success handler

* use helper for _handle_logging_vertex_collected_chunks

* enforce vertex streaming to use sse for streaming

* test test_basic_vertex_ai_pass_through_streaming_with_spendlog

* fix type hints

* add comment

* fix linting

* add pass through logging unit testing
2024-11-21 19:36:03 -08:00
Krish Dholakia
fac3b2ee42
Add pyright to ci/cd + Fix remaining type-checking errors (#6082)
* fix: fix type-checking errors

* fix: fix additional type-checking errors

* fix: additional type-checking error fixes

* fix: fix additional type-checking errors

* fix: additional type-check fixes

* fix: fix all type-checking errors + add pyright to ci/cd

* fix: fix incorrect import

* ci(config.yml): use mypy on ci/cd

* fix: fix type-checking errors in utils.py

* fix: fix all type-checking errors on main.py

* fix: fix mypy linting errors

* fix(anthropic/cost_calculator.py): fix linting errors

* fix: fix mypy linting errors

* fix: fix linting errors
2024-10-05 17:04:00 -04:00
Ishaan Jaff
96fa9d46f5 fix case when gemini is used 2024-09-10 17:06:45 -07:00
Ishaan Jaff
1c6f8b1be2 fix vertex use async func to set auth creds 2024-09-10 16:12:18 -07:00
Ishaan Jaff
c8fc453b75 fix pass through construct_target_url when vertex_proj is None 2024-09-02 12:51:30 -07:00
Krish Dholakia
e0d81434ed
LiteLLM minor fixes + improvements (31/08/2024) (#5464)
* fix(vertex_endpoints.py): fix vertex ai pass through endpoints

* test(test_streaming.py): skip model due to end of life

* feat(custom_logger.py): add special callback for model hitting tpm/rpm limits

Closes https://github.com/BerriAI/litellm/issues/4096
2024-09-01 13:31:42 -07:00
Ishaan Jaff
50014aa78e vertex forward all headers from vertex 2024-08-30 11:05:23 -07:00
Ishaan Jaff
9fb188a3b6 vertex add vertex endpoints 2024-08-29 15:49:25 -07:00
Ishaan Jaff
0e1d3804ff refactor vertex endpoints to pass through all routes 2024-08-21 17:08:42 -07:00
Ishaan Jaff
0d662368ac add native cachedContents endpoint 2024-08-08 15:32:07 -07:00
Ishaan Jaff
e73eb19678 docs default vertex 2024-08-03 18:09:48 -07:00
Ishaan Jaff
c8438715af add vertex ai countTokens endpoint 2024-08-03 17:34:10 -07:00
Ishaan Jaff
2d5c57e545 add vertex embeddings endpoints 2024-08-03 17:26:49 -07:00
Ishaan Jaff
c98733863a add vertex generateContent 2024-08-03 17:17:54 -07:00
Ishaan Jaff
8051781af0 working code for vertex ai routes 2024-08-03 16:56:33 -07:00
Ishaan Jaff
64c008045f use native endpoints 2024-08-03 16:52:43 -07:00