In-progress: e2e notebook with partial Eval integration

docs/notebooks/nvidia/tmp/sample_test_data/testing/testing.jsonl

@@ -87,4 +87,4 @@
 {"prompt": "What company owns XHDTV-TV and XHAS-TV? Due to the ratio of U.S. and Mexican-licensed stations, San Diego is the largest media market in the United States that is legally unable to support a television station duopoly between two full-power stations under FCC regulations, which disallow duopolies in metropolitan areas with fewer than nine full-power television stations and require that there must be eight unique station owners that remain once a duopoly is formed (there are only seven full-power stations on the California side of the San Diego-Tijuana market).[citation needed] Though the E. W. Scripps Company owns KGTV and KZSD-LP, they are not considered a duopoly under the FCC's legal definition as common ownership between full-power and low-power television stations in the same market is permitted regardless to the number of stations licensed to the area. As a whole, the Mexico side of the San Diego-Tijuana market has two duopolies and one triopoly (Entravision Communications owns both XHAS-TV and XHDTV-TV, Azteca owns XHJK-TV and XHTIT-TV, and Grupo Televisa owns XHUAA-TV and XHWT-TV along with being the license holder for XETV-TV, which is run by California-based subsidiary Bay City Television). Answer: ", "ideal_response": "Entravision Communications", "category": "default", "source": ""}
 {"prompt": "Besides logic and epistemology, what else did Principia Mathematica connect? The ultimate substantive legacy of Principia Mathematica is mixed. It is generally accepted that Kurt G\u00f6del's incompleteness theorem of 1931 definitively demonstrated that for any set of axioms and inference rules proposed to encapsulate mathematics, there would in fact be some truths of mathematics which could not be deduced from them, and hence that Principia Mathematica could never achieve its aims. However, G\u00f6del could not have come to this conclusion without Whitehead and Russell's book. In this way, Principia Mathematica's legacy might be described as its key role in disproving the possibility of achieving its own stated goals. But beyond this somewhat ironic legacy, the book popularized modern mathematical logic and drew important connections between logic, epistemology, and metaphysics. Answer: ", "ideal_response": "metaphysics", "category": "default", "source": ""}
 {"prompt": "What types of tools did early farmers use for crops? Neolithic people were skilled farmers, manufacturing a range of tools necessary for the tending, harvesting and processing of crops (such as sickle blades and grinding stones) and food production (e.g. pottery, bone implements). They were also skilled manufacturers of a range of other types of stone tools and ornaments, including projectile points, beads, and statuettes. But what allowed forest clearance on a large scale was the polished stone axe above all other tools. Together with the adze, fashioning wood for shelter, structures and canoes for example, this enabled them to exploit their newly won farmland. Answer: ", "ideal_response": "sickle blades and grinding stones", "category": "default", "source": ""}
-{"prompt": "People of what ethnicity were thrown out of Shusha? Gorbachev refused to make any changes to the status of Nagorno Karabakh, which remained part of Azerbaijan. He instead sacked the Communist Party Leaders in both Republics \u2013 on May 21, 1988, Kamran Baghirov was replaced by Abdulrahman Vezirov as First Secretary of the Azerbaijan Communist Party. From July 23 to September 1988, a group of Azerbaijani intellectuals began working for a new organization called the Popular Front of Azerbaijan, loosely based on the Estonian Popular Front. On September 17, when gun battles broke out between the Armenians and Azerbaijanis near Stepanakert, two soldiers were killed and more than two dozen injured. This led to almost tit-for-tat ethnic polarization in Nagorno-Karabakh's two main towns: The Azerbaijani minority was expelled from Stepanakert, and the Armenian minority was expelled from Shusha. On November 17, 1988, in response to the exodus of tens of thousands of Azerbaijanis from Armenia, a series of mass demonstrations began in Baku's Lenin Square, lasting 18 days and attracting half a million demonstrators. On December 5, 1988, the Soviet militia finally moved in, cleared the square by force, and imposed a curfew that lasted ten months. Answer: ", "ideal_response": "Armenian", "category": "default", "source": ""}
+{"prompt": "People of what ethnicity were thrown out of Shusha? Gorbachev refused to make any changes to the status of Nagorno Karabakh, which remained part of Azerbaijan. He instead sacked the Communist Party Leaders in both Republics \u2013 on May 21, 1988, Kamran Baghirov was replaced by Abdulrahman Vezirov as First Secretary of the Azerbaijan Communist Party. From July 23 to September 1988, a group of Azerbaijani intellectuals began working for a new organization called the Popular Front of Azerbaijan, loosely based on the Estonian Popular Front. On September 17, when gun battles broke out between the Armenians and Azerbaijanis near Stepanakert, two soldiers were killed and more than two dozen injured. This led to almost tit-for-tat ethnic polarization in Nagorno-Karabakh's two main towns: The Azerbaijani minority was expelled from Stepanakert, and the Armenian minority was expelled from Shusha. On November 17, 1988, in response to the exodus of tens of thousands of Azerbaijanis from Armenia, a series of mass demonstrations began in Baku's Lenin Square, lasting 18 days and attracting half a million demonstrators. On December 5, 1988, the Soviet militia finally moved in, cleared the square by force, and imposed a curfew that lasted ten months. Answer: ", "ideal_response": "Armenian", "category": "default", "source": ""}

docs/notebooks/nvidia/tmp/sample_test_data/validation/validation.jsonl

@@ -87,4 +87,4 @@
 {"prompt": "Birds preen often with the application of secretions from which gland? Feathers being critical to the survival of a bird, require maintenance. Apart from physical wear and tear, feathers face the onslaught of fungi, ectoparasitic feather mites and birdlice. The physical condition of feathers are maintained by preening often with the application of secretions from the preen gland. Birds also bathe in water or dust themselves. While some birds dip into shallow water, more aerial species may make aerial dips into water and arboreal species often make use of dew or rain that collect on leaves. Birds of arid regions make use of loose soil to dust-bathe. A behaviour termed as anting in which the bird encourages ants to run through their plumage is also thought to help them reduce the ectoparasite load in feathers. Many species will spread out their wings and expose them to direct sunlight and this too is thought to help in reducing fungal and ectoparasitic activity that may lead to feather damage. Answer: ", "completion": "preen gland"}
 {"prompt": "Who was the first person to carry the torch?  Greece: On March 24, 2008, the Olympic Flame was ignited at Olympia, Greece, site of the ancient Olympic Games. The actress Maria Nafpliotou, in the role of a High Priestess, ignited the torch of the first torchbearer, a silver medalist of the 2004 Summer Olympics in taekwondo Alexandros Nikolaidis from Greece, who handed the flame over to the second torchbearer, Olympic champion in women's breaststroke Luo Xuejuan from China. Following the recent unrest in Tibet, three members of Reporters Without Borders, including Robert M\u00e9nard, breached security and attempted to disrupt a speech by Liu Qi, the head of Beijing's Olympic organising committee during the torch lighting ceremony in Olympia, Greece. The People's Republic of China called this a \"disgraceful\" attempt to sabotage the Olympics. On March 30, 2008 in Athens, during ceremonies marking the handing over of the torch from Greek officials to organizers of the Beijing games, demonstrators shouted 'Free Tibet' and unfurled banners; some 10 of the 15 protesters were taken into police detention. After the hand-off, protests continued internationally, with particularly violent confrontations with police in Nepal. Answer: ", "completion": "Alexandros Nikolaidis"}
 {"prompt": "What nationality was Friedrich Schleiermacher? Many non-transparent-translation theories draw on concepts from German Romanticism, the most obvious influence being the German theologian and philosopher Friedrich Schleiermacher. In his seminal lecture \"On the Different Methods of Translation\" (1813) he distinguished between translation methods that move \"the writer toward [the reader]\", i.e., transparency, and those that move the \"reader toward [the author]\", i.e., an extreme fidelity to the foreignness of the source text. Schleiermacher favored the latter approach; he was motivated, however, not so much by a desire to embrace the foreign, as by a nationalist desire to oppose France's cultural domination and to promote German literature. Answer: ", "completion": "German"}
-{"prompt": "What would an omnidirectional antenna look like if plotted? The radiation pattern of an antenna is a plot of the relative field strength of the radio waves emitted by the antenna at different angles. It is typically represented by a three-dimensional graph, or polar plots of the horizontal and vertical cross sections. The pattern of an ideal isotropic antenna, which radiates equally in all directions, would look like a sphere. Many nondirectional antennas, such as monopoles and dipoles, emit equal power in all horizontal directions, with the power dropping off at higher and lower angles; this is called an omnidirectional pattern and when plotted looks like a torus or donut. Answer: ", "completion": "donut"}
+{"prompt": "What would an omnidirectional antenna look like if plotted? The radiation pattern of an antenna is a plot of the relative field strength of the radio waves emitted by the antenna at different angles. It is typically represented by a three-dimensional graph, or polar plots of the horizontal and vertical cross sections. The pattern of an ideal isotropic antenna, which radiates equally in all directions, would look like a sphere. Many nondirectional antennas, such as monopoles and dipoles, emit equal power in all horizontal directions, with the power dropping off at higher and lower angles; this is called an omnidirectional pattern and when plotted looks like a torus or donut. Answer: ", "completion": "donut"}