{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# k-Nearest Neighbors"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Introduction"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"k-nearest neighbors (*kNN*) is an intuitively simple algorithm in which the label (in classification) or continuous value (in regression) of an unknown test data point is determined using the closest *k* points to it in a training dataset. The distance between data points is often calculated using [Euclidean distance](https://en.wikipedia.org/wiki/Euclidean_distance), although there are other possible choices.\n",
"\n",
"```{note}\n",
"*kNN* is sometimes called a \"lazy\" model because it leaves all the computational work until testing time. Training simply involves storing the training dataset, whereas testing an unknown data point involves searching the training dataset for the *k* nearest neighbors.\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Animation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The animation below shows an example of a *kNN* classifier, where an unknown data point is classified using the most common class amongst the *k* nearest neighbors (meassured by Euclidean distance). In the example, there are two features (*X1* and *X2*) and the response (*Y*). Choose different values of *k* from the drop down menu and observe how the classification of the unknown data point changes.\n",
"\n",
"```{note}\n",
"In the regression setting, it is common to assign the average value of the *k* nearest neighbors to the test data point.\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [
{
"data": {
"text/html": [
" \n",
" "
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
"
\n",
" \n",
"