Vario-Eta Learning The update rule is Δwti=−11T∑(gti−gi)2gti\Delta w_t^i=-\frac{1}{\sqrt{\frac{1}{T} \sum\left(g_t^i-g^i\right)^2}} g_t^iΔwti=−T1∑(gti−gi)21gti The sum is computed using Exponential Smoothing.