boundlab.ibp.softmax#
Softmax via the DeepT shift-invariant decomposition.
Dividing numerator and denominator by \(e^{\nu_i}\) turns softmax into
\[\sigma_i(\nu) = \frac{e^{\nu_i}}{\sum_j e^{\nu_j}}
= \frac{1}{\sum_j e^{\nu_j - \nu_i}},\]
a composition of pairwise differences, exp, a reduce-sum, and one
reciprocal — no product of two abstract values, and the differences keep
the exponent range centered so exp stays well-conditioned.
Classes
Rewrite softmax as \(1 / \sum_j e^{\nu_j - \nu_i}\) over the last axis. |