Kernel Functions

Kernel functions assign weights to nearby samples in nonparametric statistics — most notably in kernel density estimation and the Nadaraya-Watson regression smoother. The choice of kernel controls how quickly weight falls off with distance and whether the kernel has compact support, both of which affect bias, variance, and computational cost. While the Gaussian kernel is the textbook default, the compact-support Epanechnikov kernel is provably mean-squared-error optimal in standard settings.

The plot compares five common kernels: Gaussian, Epanechnikov, quartic (biweight), tricube, and uniform. The four compact kernels are drawn over -1 \le t \le 1, where they vanish at the boundary; the Gaussian is drawn over the wider domain -3.5 to 3.5 because its tails never reach zero exactly. The visual contrast makes the tradeoff explicit: smoother kernels taper gracefully, the uniform kernel gives equal weight inside its support and a sharp cliff at the boundary, and the Gaussian trades compact support for an infinitely smooth shape.