Maximum Likelihood Estimation

From all possible ϕ\phi parameters we will select the ones which most likely generated the training set. (selecting the maximum depending on the Dataset)

arg⁡max⁡hiP(d∣hi)\underset{h_i}{\arg\max}\quad P(d\mid h_i)

= MAP Learning for a uniform Prior.

This method uses the Log-Likelihood function. It has to be maximized to get to its minimum (negative sign).

Solve with Newton Method:

So we have:

θk+1=θk−∇2L(θk)−1∇L(θk)\theta^{k+1}=\theta^k-\nabla^2L(\theta^k)^{-1}\nabla L(\theta^k)

So basically current parameters minus inverse of the hessian matrix times the Gradient.

One can use different methods to calculate the minimum: