quantization

What limits local LLM inference

A practical way to reason about model size, number formats, memory bandwidth, compute, and the software between them.

• inference
• hardware
• quantization