Softmax

A smoothed variation of the maximization function that is derivable which is important for Backpropagation.

σ:R×Rk→Rk\sigma: \mathbb{R} \times \mathbb{R}^k \rightarrow \mathbb{R}^k σβ(z)i:=eβzi∑j=1keβzj\sigma_\beta(z)_i:=\frac{e^{\beta z_i}}{\sum_{j=1}^k e^{\beta z_j}}

where z=⟨z1,…,zk⟩,i∈{1,…,k}z=\left\langle z_1, \ldots, z_k\right\rangle, i \in\{1, \ldots, k\}

and eβe^{\beta} is called the base. For β=1\beta=1 the function is also called the standard softmax function.