Labs · Size and tokens

One distribution, one token

Greedy, temperature, top-k and top-p on one next-token distribution, with a thousand seeded draws to show what each keeps and what each throws away.

pi=ezi/T∑j∈Sezj/Tp_i = \frac{e^{z_i/T}}{\sum_{j \in S} e^{z_j/T}}12 candidates · 1,000 draws

Two strong candidates, four plausible, six unlikely: the tail is where the strategies part.

What each strategy keeps, and what it draws

Reads /api/labs/decoding — loading

Computing…

One distribution, one token — Labs — TransformerLab