Generalization Loss

The generalization loss for a hypothesis hh from a given Hypothesis Space with respect to a Loss Function LL and examples E\mathcal{E} is

GenLoss⁡L(h):=∑(x,y)∈EL(y,h(x))⋅P(x,y).\operatorname{GenLoss}_L(h):=\sum_{(x, y) \in \mathcal{E}} L(y, h(x)) \cdot P(x, y).

We can find the best hypothesis

h∗:=argmin⁡h∈HGenLoss⁡L(h).h^*:=\underset{h \in \mathcal{H}}{\operatorname{argmin}} \operatorname{GenLoss}_L(h).

As P(X,Y)P(X, Y) is unclear in practice we can only estimate the loss with the Empirical Loss.