TranslationInvariantKernels

Given a Gaussian process $GP (\mu, K)$, where $\mu : \mathbb{R}^n \to \mathbb{R}$ represents the mean function and $K : \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ represents the covariance function or kernel of the Gaussian process, the kernel $K$ is said to be translation invariant if and only if there exists a function $k : \mathbb{R}^n \to \mathbb{R}$ such that for any $x, x' \in \mathbb{R}^n$, the covariance function $K$ satisfies the following condition: $$K (x, x') = k (x - x')$$ Here, $x - x'$ denotes the vector difference between $x$ and $x'$, implying that the covariance between any two points depends solely on their relative displacement in the input space $\mathbb{R}^n$. The function $k$ is often referred to as the stationary kernel function, and it encapsulates the essence of translation invariance: the statistical properties of the process do not change when shifted in the input space.

A common example of a translation invariant kernel is the squared exponential kernel (also known as the radial basis function or RBF kernel), defined as: $$k (x - x') = \sigma^2 \exp \left( - \frac{1}{2 \ell^2} |x - x' |^2 \right)$$ where $\sigma^2$ is the variance parameter, $\ell$ is the length-scale parameter, and $|x - x' |$ is the Euclidean distance between $x$ and $x'$.