Linear model

A linear model consists of a matrix AA which contains all coefficients. We can then write our set of possible regression functions like this:

fγ(t)=Aγf_{\gamma}(t) =A\cdot\gamma

Of course there are also other models which can include non-linearities like:

Simple linear model

This is how a simple two dimensional linear model can look like:

f(x)=w0x0+w1x1=yf(x) = w_0 \cdot x_0 + w_1 \cdot x_1 = y

The input is the x vector with x0x_0 and x1x_1 as values. The same applies to the weights. Finding the best weights to describe yy with our function is called Fitting.

Usually a residual error ϵ\epsilon is part of the equation. A lot of times it has a Gaussian distribution.

In vector notation the general linear model looks like this:

f(x)=wTx+ϵ=yf(x) = w^Tx + \epsilon = y

Now we have to optimize the model parameters: