GPU: the chip that runs artificial intelligence
A GPU is a chip designed to perform a huge number of simple calculations at the same time. Originally built to draw video game graphics, it has become the engine for training and running AI models.
Data as of October 22, 2026
Key figures
| Level | Advanced |
|---|---|
| Origin | 1999 |
| Introduced by | NVIDIA (selon sa chronologie officielle) |
What a GPU is
A GPU is a chip that does a vast number of small calculations at once, whereas a regular processor does a few very quickly.
A computer’s central processor (CPU) has a handful of powerful cores designed to juggle varied tasks. A GPU (graphics processing unit) lines up thousands of simpler ones that apply the same operation to lots of data at the same time. That design was meant for computing the color of millions of pixels. According to its official timeline, NVIDIA claims it invented the GPU in 1999, and in 2006 it opened its chips to scientific computing with the CUDA architecture.
As it happens, a neural network boils down mostly to multiplying large matrices: exactly the kind of massively parallel work GPUs are good at.
A concrete example
When you ask an assistant a question, your text is split into tokens and sent to a data center. There, one or more GPUs load the model into their VRAM and compute the answer token by token: that is inference. A large model often does not fit on a single card: Hugging Face’s documentation notes that a 70-billion-parameter model needs about 140 GB in 16-bit precision, while the A100 and H100 data center cards it mentions offer 80 GB. The model then has to be split across several GPUs.
Why it matters
The GPU is AI’s scarce resource. The training compute behind a frontier model ties up a large number of chips for a long stretch of time, which explains the huge investment in data centers. For users, the price of a subscription or an API largely reflects the GPU time consumed. For local AI, your graphics card decides what you can run, which is why quantization matters so much.
Common myths
- “Raw compute is what matters most.” For generating text, memory size and bandwidth are often the bottleneck before raw power.
- “A gaming GPU is useless for AI.” Consumer cards run plenty of quantized open models.
- “NVIDIA is the only option.” NVIDIA dominates, helped by its software ecosystem, but other manufacturers and dedicated accelerators exist.
Related terms: VRAM, inference server, training compute and deep learning.
Related terms
- VRAM (video memory) · The memory built into a graphics card, where an AI model must fit to run fast; it is often what limits what you can run locally.
- Training compute (FLOP) · The total number of arithmetic operations, measured in FLOP, needed to train an AI model.
- Inference · The moment a trained model is actually used to produce an output: every question you send to an AI triggers an inference run.
- Inference server · Software that loads an AI model into memory and answers requests through an API, while optimizing throughput and GPU memory.
- Deep learning · Machine learning that uses neural networks with many layers; it is the technique behind today's chatbots, image and voice AI.
- Quantization · A technique that stores a model's weights with fewer bits (8, 4 or even fewer) so it needs far less memory to run.
Full glossary
Frequently asked questions
Do I need a GPU to use ChatGPT or Claude?
No. These assistants run on GPUs in their providers' data centers; your phone or computer only sends the question and displays the answer. A personal GPU is only useful if you want to run a model locally.
What should I look for in a GPU for local AI?
First, the amount of video memory (VRAM), which determines how large a model the card can load. Then memory bandwidth, which affects generation speed, and software compatibility.
Are GPUs the only chips used for AI?
No. There are specialized accelerators such as Google's TPUs, and regular processors (CPUs) can run small models. The vLLM documentation, for example, lists NVIDIA and AMD GPUs, CPUs and TPUs among supported hardware.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.