burnt-synapse

← back to blog

mtp: 60 t/s when the text is predictable

22 August 2026

qwenllama.cppmtp

the box does 35 t/s of generation. with mtp — multi-token prediction — it does 60+ when the next few tokens are easy to guess.

the idea: instead of predicting one token per forward pass, the model carries a small head that predicts the next three tokens at once. if they're right, the step is worth triple; if not, you drop the tail and continue.

that's why it's scenario-dependent. in predictable scenes — boilerplate code, repetitive output, formulaic prose — the 3-token speculation almost always lands and tg jumps past 60. in open-ended writing, where every token is a coin flip, the head guesses wrong and you're back near 35.

same four cards, same q6_k model, same everything. the only difference is that the model is allowed to bet on three tokens at once.