qwen3.8 27b on $500 of scrap

qwen3.8 27b — q8 weights, f16 kv cache — doing 200 t/s of prompt processing and 30 t/s of generation on one box of $500 scrap. no gpu in sight.

the setup

the model runs at q8 — the quant where you stop noticing the quant — with an f16 kv cache. i refuse to quantize the cache: on long contexts the f16 cache is the difference between the model keeping its grip and slowly forgetting the first half of the conversation, and the extra ram is the least of your problems.

the hardware

one box, running one model, with all of this in it at the same time:

• 2× 2014 xeon — e5 v3, a decade of server leftovers that cost peanuts.
• 4× cmp50hx — the modern ones.

six chips, one qwen, no gpu. about $500 of hardware, all-in. that's the whole point: 27b used to mean rent something or buy the card, and now it means dig through the drawer.

the numbers

box	prompt processing	generation
2× 2014 xeon + 4× cmp50hx	200 t/s	30 t/s

why it matters

30 t/s is "faster than i can read" for generation, and 200 t/s on the prompt side means the model swallows a big context before you stop typing. on a box with a decade-old xeon in it.

the "27b depends on the hardware" era is over. it depends on whether you quantized the weights to q8 and kept the cache honest.