Outlier Detection with Parametric Statistical Method

Example for Gaussian Process

We assume that data is generated from some distribution (e.g. Normal Distribution) Learn parameters from input data and identify points with low probability as Outliers.

Use Maximum Likelihood Estimation to estimate parameters of Distribution. $\mathcal{L}(\theta \mid X)=\mathcal{L}\left(\mu ,\sigma \mid x_1,\dots ,x_n\right)={\prod }_{i=1}^n\frac{1}{\sqrt{2\pi {\sigma }^2}}{\mathrm{e}}^{-\frac{{\left(x_i-\mu \right)}^2}{2{\sigma }^2}}$ Now we want to find good parameters to maximize this function.

General Procedure

1. Generate gradients for each parameter
2. Solve each Gradient by setting it to zero

Using the Log-Likelihood of the function makes it easier to calculate the derivatives.

Results

• Expectation is Arithmetic Mean
• Standard Deviation is square root of Sample Variance

Now we can use the property of a Normal Distribution that 3 Standard Deviations from the mean contain 99.7 percent of the data to calculate outliers.

Grubbs Test

Grubbs Test tests the hypothesis if the data contains no outliers or exactly one outlier.

For multivariate data:

Transform multivariate task into univariate problem.

Methods:

• Mahalanobis Distance

For normally distributed data we can use the chi-squared Test Statistic ${\chi }^2={\sum }_{i=1}^n\frac{x_i-\bar{x}}{\bar{x}}$ which is large if there is an outlier.

Mixture of multiple distributions

For example every datapoint can be generated by two normal distributions: $\mathrm{P}\left(\mathbf{x}\mid {\mathcal{N}}_1,{\mathcal{N}}_2\right)=f_{{\mathcal{N}}_1}\left(\mathbf{x}\mid {\mu }_1,{\sigma }_1\right)+f_{{\mathcal{N}}_2}\left(\mathbf{x}\mid {\mu }_2,{\sigma }_2\right)$ Use Gaussian Mixture Expectation Maximization to estimate the parameters from the data.