add everyting for docs

This commit is contained in:
ishaan-jaff 2023-07-29 07:00:13 -07:00
parent 36eaaa7d36
commit d400bccb15
1015 changed files with 185353 additions and 0 deletions

View file

@ -0,0 +1,166 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "3dd292b1-9a73-4ea8-af19-5fa6e3c1a62a",
"metadata": {},
"source": [
"# Brave Search\n",
"\n",
"\n",
">[Brave Search](https://en.wikipedia.org/wiki/Brave_Search) is a search engine developed by Brave Software.\n",
"> - `Brave Search` uses its own web index. As of May 2022, it covered over 10 billion pages and was used to serve 92% \n",
"> of search results without relying on any third-parties, with the remainder being retrieved \n",
"> server-side from the Bing API or (on an opt-in basis) client-side from Google. According \n",
"> to Brave, the index was kept \"intentionally smaller than that of Google or Bing\" in order to \n",
"> help avoid spam and other low-quality content, with the disadvantage that \"Brave Search is \n",
"> not yet as good as Google in recovering long-tail queries.\"\n",
">- `Brave Search Premium`: As of April 2023 Brave Search is an ad-free website, but it will \n",
"> eventually switch to a new model that will include ads and premium users will get an ad-free experience.\n",
"> User data including IP addresses won't be collected from its users by default. A premium account \n",
"> will be required for opt-in data-collection.\n"
]
},
{
"cell_type": "markdown",
"id": "26f0888e-3f3e-4b82-ac4a-2df6feeccbe0",
"metadata": {},
"source": [
"## Installation and Setup\n",
"\n",
"To get access to the Brave Search API, you need to [create an account and get an API key](https://api.search.brave.com/app/dashboard).\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "d7d7be09-58bd-47d7-bf1b-33964564f777",
"metadata": {},
"outputs": [],
"source": [
"api_key = \"...\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b3ac92df-6ff0-4dbb-b32b-a7dc140c48ef",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import BraveSearchLoader"
]
},
{
"cell_type": "markdown",
"id": "7f483caf-58ef-4138-975a-5b783559dc1b",
"metadata": {},
"source": [
"## Example"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "766634cf-3bc7-4656-939a-cafa218807a6",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loader = BraveSearchLoader(\n",
" query=\"obama middle name\", api_key=api_key, search_kwargs={\"count\": 3}\n",
")\n",
"docs = loader.load()\n",
"len(docs)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "f1fcc9f1-cbdc-46b3-89d3-80311d557dc6",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'title': \"Obama's Middle Name -- My Last Name -- is 'Hussein.' So?\",\n",
" 'link': 'https://www.cair.com/cair_in_the_news/obamas-middle-name-my-last-name-is-hussein-so/'},\n",
" {'title': \"What's up with Obama's middle name? - Quora\",\n",
" 'link': 'https://www.quora.com/Whats-up-with-Obamas-middle-name'},\n",
" {'title': 'Barack Obama | Biography, Parents, Education, Presidency, Books, ...',\n",
" 'link': 'https://www.britannica.com/biography/Barack-Obama'}]"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"[doc.metadata for doc in docs]"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "601bfd77-03d3-468e-843f-2523d5e215bd",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['I wasnt sure whether to laugh or cry a few days back listening to radio talk show host Bill Cunningham repeatedly scream Barack <strong>Obama</strong><strong>s</strong> <strong>middle</strong> <strong>name</strong> — my last <strong>name</strong> — as if he had anti-Muslim Tourettes. “Hussein,” Cunningham hissed like he was beckoning Satan when shouting the ...',\n",
" 'Answer (1 of 15): A better question would be, “Whats up with <strong>Obama</strong>s first <strong>name</strong>?” President Barack Hussein <strong>Obama</strong>s fathers <strong>name</strong> was Barack Hussein <strong>Obama</strong>. He was <strong>named</strong> after his father. Hussein, <strong>Obama</strong><strong>s</strong> <strong>middle</strong> <strong>name</strong>, is a very common Arabic <strong>name</strong>, meaning &quot;good,&quot; &quot;handsome,&quot; or ...',\n",
" 'Barack <strong>Obama</strong>, in full Barack Hussein <strong>Obama</strong> II, (born August 4, 1961, Honolulu, Hawaii, U.S.), 44th president of the United States (200917) and the first African American to hold the office. Before winning the presidency, <strong>Obama</strong> represented Illinois in the U.S.']"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"[doc.page_content for doc in docs]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "74a6ba54-9e48-4bac-ab9b-03eabd19eb81",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}