update docs

This commit is contained in:
ishaan-jaff 2023-08-10 18:58:39 -07:00
parent d288028f24
commit b47b6bf5a1

View file

@ -0,0 +1,96 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Using Google Palm (VertexAI) with liteLLM \n",
"### chat-bison & chat-bison@001"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install litellm==0.1.379 # use 0.1.379 or after for VertexAI"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Set VertexAI Configs\n",
"Vertex AI requires the following:\n",
"* `vertex_project` - Your Project ID\n",
"* `vertex_location` - Your Vertex AI region\n",
"Both can be found on: https://console.cloud.google.com/\n",
"\n",
"VertexAI uses Application Default Credentials, see https://cloud.google.com/docs/authentication/external/set-up-adc for more information on setting this up\n",
"\n",
"NOTE: VertexAI requires you to set `application_default_credentials.json` \n"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"# set you Vertex AI configs\n",
"import litellm\n",
"from litellm import embedding, completion\n",
"\n",
"litellm.vertex_project = \"hardy-device-386718\"\n",
"litellm.vertex_location = \"us-central1\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Call VertexAI using liteLLM"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"user_message = \"what is liteLLM \"\n",
"messages = [{ \"content\": user_message,\"role\": \"user\"}]\n",
"\n",
"# chat-bison or chat-bison@001 supported by Vertex AI (As of Aug 2023)\n",
"response = completion(model=\"chat-bison\", messages=messages)\n",
"print(response)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.6"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}