Labs · Size and tokens

Parameters, FLOPs and memory

A model's size from its dials — every parameter counted by the platform's own derivation, the FLOPs a token costs, and what training it holds in memory.

N≈12 L d2+V dN \approx 12\,L\,d^2 + V\,dthis platform's model, up to a 7B decoder

The numpy engine as served: d 16, 2 heads, 2 layers, on the bundled corpus.

What these dials cost

Reads /api/labs/complexity — loading

Computing…

Parameters, FLOPs and memory — Labs — TransformerLab