Building & Running AI beginner

GPU

Also called: graphics processing unit

The parallel processor that made deep learning practical, and the scarcest resource in the industry.

GPUs perform the massive matrix multiplications behind training and inference thousands of times faster than a CPU. Their memory capacity sets the ceiling on model size, which is why quantization matters. Supply and price of GPUs shape the whole market.

In practice: Whether a model fits in 24GB of VRAM decides if it runs on your desk.

Related terms

Where this comes up