An extremal type of cross$-$validation is n$-$fold cross$-$validation on a training set of size n$.$ If we want

to estimate the error of k$-$NN$,$ this amounts to classifying each training point by running k$-$NN on

the remaining n $1$ points, and then looking at the fraction of mistakes made. It is commonly called

leave$-$one$-$out cross$-$validation $($LOOCV$).$