What is synthetic data in AI? Definition and examples

Level	Intermediate

A simple definition

A model learns from examples: its training data. When examples are missing, you can manufacture them. Synthetic data means text, images, tables or dialogues generated by a program, these days often a large language model, to resemble real data.

A concrete example

A small company wants an assistant that sorts its support tickets into ten categories, but it only has 200 labeled tickets. It asks a large model to write 5,000 varied tickets: rushed or polite customers, typos, mixed requests. After spot-checking a sample, it uses this set for fine-tuning a small language model, then measures the result on its real tickets. Those real tickets are kept aside for this final test: they are the only reliable way to check that the model has learned the actual task rather than the quirks of the generator.

The same idea works at scale. In June 2023, a research team introduced phi-1, a 1.3-billion-parameter code model trained in part on about 1 billion tokens of textbooks and exercises generated with GPT-3.5, alongside 6 billion tokens selected from the web.

Why it matters

Synthetic data addresses three problems: scarcity (rare cases, under-represented languages), cost (human labeling is slow and expensive) and privacy (avoiding exposure of real customer records). It is also central to distillation, where a large model produces the examples used to train a smaller one.

Common mistakes and myths

“The more, the better.” A large volume of repetitive examples teaches little. Variety and accuracy matter more than quantity.
“A model can keep training on its own output forever.” A study by Shumailov and co-authors, published in 2023, shows that training models generation after generation on generated data degrades their output: this is model collapse.
“Synthetic means risk-free.” The generator model’s errors and biases carry over into the data, then into the trained model.

Related terms: training data, distillation, fine-tuning, model collapse.

Related terms

Training data · The examples (text, images, audio and more) an AI model learns from; they shape what the model knows and where it goes wrong.

Model collapse · The gradual degradation of models trained, generation after generation, on content produced by other models rather than by people.

Knowledge distillation · A technique that trains a small student model to imitate the outputs of a large teacher model, so it becomes lighter and cheaper to run.

Fine-tuning · Continuing the training of an already trained model on a small, targeted set of examples to adapt it to a task, style or output format.

Small language model (SLM) · A compact language model, light enough to run on a laptop or phone, that performs well on focused tasks rather than everything at once.

Frequently asked questions

Is synthetic data fake?

It is manufactured, not misleading: plausible examples created to mimic a real situation. Its value depends on how accurate and varied it is, and on the quality checks applied before use.

Can it fully replace real data?

Rarely. It usually complements a base of real data, which is also needed to check that the model works on genuine cases. A final test on real examples remains essential.

Is synthetic data anonymous?

Not automatically. A generator model can reproduce fragments of the data it was trained on. If personal data is involved, have the approach reviewed by whoever handles compliance.

How is it generated?

Most often by asking a large language model to produce examples to a precise brief (topic, tone, difficulty, format), then filtering the output, sometimes with a second model that grades it.

Synthetic data: training AI on manufactured examples

Key figures