Fig. 2

Example of how an algorithm’s behavior and results change when the hyper-parameter changes, for the the k-nearest neighbors method [20] (image adapted from [72]). a In this example, there are six blue square points and five red triangle points in the Euclidean space. A new point (the green circle) enters the space, and k-NN has to decide to which category to assign it (red triangle or blue square). b If we set the hyper-parameter k=3, the algorithm considers only the three points nearest to the new green circle, and assigns the green circle to the red triangle category (two red triangles versus one blue square). c Likewise, if we set the hyper-parameter k=4, the algorithm considers only the four points nearest to the new green circle, and assigns the green circle again to the red triangle category (the two red triangles are nearer to the green circle than the two blue squares). d However, if we set the hyper-parameter k=5, the algorithm considers only the five points nearest to the new green circle, and assigns the green circle to the blue square category (three blue squares versus two red triangles)