Synthetic data: the risks and limits that no one puts on the sales slide

Synthetic data has become the standard answer to two real problems: privacy and lack of data. The promise is tempting, train without exposing anyone, test without copying production, generate as much mass as you want. Like any promise that's too good, the interesting part is what the supplier doesn't put on the slide.

Synthetic data is not real data with built-in privacy. It's a statistical imitation, and imitation has a limit. Whoever treats the synthetic as a perfect substitute for the real exchanges a visible risk (leaking personal data) for an invisible risk (making decisions based on a world that does not exist). The second is worse precisely because it doesn't show up until late.

The risk of learning the world wrong

A model learns the world you show it. If the world is synthetic, it learns the synthetic version, with all the shortcuts and simplifications that the generator built in. The problem is that these shortcuts are not labeled.

When synthetic data captures the structure of the real thing well, great. When it captures poorly, the model becomes confident about patterns that only exist in the simulation. He's right in the laboratory and wrong in the world, and the difference between the two environments is exactly the part that the generator didn't know how to reproduce.

The classic case is the rare event. Fraud, critical failure, unusual illness, what matters is often the exception, not the average. Generators are good at reproducing the typical behavior and bad at reproducing the tail. You generate a million synthetic transactions and unintentionally teach the model that fraud almost doesn't happen the way it actually does.

Bias does not disappear in the copy, it multiplies

There is a comfortable fantasy that generating synthetic data “cleans” the data. Doesn't clean. The generator learns from the real data, and everything that was crooked in the real data tends to appear in the synthetic, sometimes amplified.

If the original database underrepresents a group, the synthetic data will also underrepresent it, with the added bonus of appearing neutral. The bias gets a new coat of paint: it is now in an "artificial" dataset, which gives a false sense of impartiality. The injustice continues, it has only become more difficult to point the finger at.

For those who work with decisions that affect people, credit, screening, prioritization of public service, this is serious. LGPD Compliance takes care not to expose the individual, but does not guarantee that the model trained above is fair. They are two different problems, and solving the first does not solve the second.

The re-identification that the synthetic should avoid

The privacy argument for synthetic data is strong: as there is no real person there, there is no one to re-identify. In theory. In practice, it depends on how the data was generated.

A generator that overly decorates the original set can reproduce real excerpts almost intact, especially the extreme and unique cases, precisely those that are easiest to reidentify. The patient's record with the rare combination of conditions may reappear in the synthetic almost the same. You thought you had anonymized it and, without realizing it, you copied what you most needed to hide.

This means that synthetic data is not private by definition. It is private when it has been carefully generated and measured judiciously. Without that measurement, "it's synthetic, so it's safe" is faith, not guarantee. And faith does not go through auditing.

Model collapse: when AI feeds on its own reflection

There is a newer, quieter risk when synthetic data becomes the main source of training: the model starts to learn from model outputs, not from the world. With each generation, diversity shrinks a little, the edges are lost, the average dominates.

It's like making a photocopy of a photocopy. The first one is great. On the twentieth, the text is still readable, but the tone is gone. Systems that feed on synthetic data without sufficient injection of reality tend to become impoverished over time, becoming more and more like themselves and less and less like what they should model.

The antidote is not to abandon the synthetic, it is to keep the real in the loop. Synthetic dice works best as a supplement and amplification, not as an exclusive diet. Anyone who cuts ties with the real world slowly loses the ability to represent it.

Where synthetics work, and where they don’t

It's worth being specific, because synthetics are neither good nor bad in the abstract, they are suitable or not for each use. It yields a lot in testing and QA: generate realistic mass to fill an approval environment, exercise flows, find bugs, without dragging production data. Here the risk is low, because the objective is to exercise the system, not to make decisions about people.

It pays off in development and demonstration, where you need something plausible and can't use the real thing. It pays to increase data when there are no examples of a specific situation and you know well how it behaves.

Where it costs a lot is in the high-risk decision trained almost only on the synthetic, and in the final validation. A model that goes into production deciding about people needs, at some point, to face real data. The synthetic accelerates the way there, it does not replace the arrival. Skipping real validation to save time is exchanging today's cost for tomorrow's incident.

The question that separates mature use from illusion

The decision is not "to use or not to use synthetic data", it is "for what, with which generator and validated against what". Anyone who answers these three questions uses the tool well. Anyone who jumps straight to "it's synthetic, so it can" is outsourcing judgment to a marketing slide.

Synthetic data is a powerful tool for privacy, speed and scale. Like any powerful tool, it fails in convincing ways: the error comes with a vengeance, with volume, with the appearance of rigor. That's why the most important limit is not technical, it's one of posture: treating the synthetic with the same healthy distrust that you would give to any source that claims to represent reality without being reality.

In the end, synthetic doesn't eliminate the need to understand your data. He increases it. Only those who deeply understand the original know how to generate a good imitation, and only those who continue to measure the distance between themselves and the world know how to trust the imitation. Anyone who loses sight of this distance has not automated privacy, they have automated error.

If you are evaluating synthetic data for an AI product or a project with sensitive data, and want to separate the real gain from the hidden risk, it's worth talking. I have other texts on the blog about data governance, privacy and responsible adoption of AI.