Softmax

What it is

Softmax is a mathematical function that converts logits into probabilities that sum to 1. It takes a list of scores and transforms them so they represent a proper probability distribution. Higher scores become higher probabilities, but all values remain positive and sum to 100%.

Why it matters

Softmax is the bridge between a model's internal scores and the probabilities used for sampling. When temperature is applied, it modifies values before softmax, changing how "peaked" or "flat" the probability distribution becomes. This is a technical detail, but it explains how temperature affects output randomness.