Softmax

Generalizes Sigmoid to multi-class settings. Converts a vector of raw scores (logits) into a probability distribution that sums to 1.

$$ \text{softmax}(x_i) = \frac{e^{x_i}}{\sum_{j} e^{x_j}} $$

Output probabilities sum to 1 across all classes

Used in multi-class classification output layers

Numerically stabilized in practice by subtracting $\max(x)$ from all logits before exponentiation