GROUND TRUTH · AI / ML

When “More Data” Makes an AI System Worse

A model does not know the difference between a real pattern and a coincidence in the data. More examples of the coincidence just make it more confident.

More training data only helps if it reflects a true, relevant pattern, otherwise it teaches the wrong lesson.

By Prashanth Jakkula · Published 18 September 2026 · Researched from job postings, engineering references, and public practitioner accounts, then written and checked by the author before publication. AI tools may help with research and editing. See our Editorial Policy for how articles are produced and reviewed, or About HireSetu for who operates it.

A very common intuition for anyone starting out in machine learning is that more training data should always make a model better, in roughly the same way that more practice problems should make a student better at a subject. This intuition is not baseless, in many real situations, more data does help, especially when a model is underfitting because it has not seen enough examples of the pattern it needs to learn. But the intuition breaks down in a specific and important way: a model learns whatever pattern is present in the data it is given, whether or not that pattern is the one you wanted it to learn, and whether or not it will still hold true outside that dataset. More data does not filter out a bad pattern. It can reinforce one.

A model cannot distinguish a real signal from a systematic bias in how the data was collected

Imagine a model trained to predict loan default risk, built on a bank's historical lending data. If that bank's past lending decisions were themselves influenced by factors that had little to do with actual repayment ability, say, a historical tendency to approve fewer loans in certain neighborhoods, for reasons unrelated to creditworthiness, the training data does not contain a clean record of "who would have repaid." It contains a record of who was approved and did repay, mixed with the effects of whatever biases shaped the original approval decisions. A model trained on more of this same data does not become less biased; it becomes a more confident, more precisely calibrated version of the same biased pattern, because from the model's perspective, that pattern is what the data consistently shows. Adding more examples of a systematically skewed dataset does not average the skew away. It can make the model fit the skew more tightly.

The intuition vs what determines whether more data helps

More data can also mean more room to memorize instead of generalize

This sounds counterintuitive at first, since overfitting is usually described as a problem of too little data relative to model complexity, but the risk works in both directions depending on what the additional data contains. If a large volume of new data introduces a lot of noisy, low-quality or mislabeled examples, a sufficiently flexible model can start fitting to that noise rather than to the genuine underlying pattern, especially if the noisy examples are not randomly distributed but share some accidental characteristic, say, a particular data-collection batch where labels were systematically applied incorrectly. The model, given enough capacity and enough repetitions of that mislabeled pattern, can learn the mislabeling as if it were signal. This is part of why experienced practitioners spend real time auditing what a large new dataset contains before assuming that adding it will help: volume alone is not a proxy for quality, and a model has no independent way to tell the difference.

Distribution shift: the data can be accurate and still be the wrong data

A more subtle version of the same problem occurs even when the new data is perfectly accurate on its own terms but represents a different underlying situation than the one the model will be used in. A fraud-detection model trained heavily on transaction patterns from one country, one time period, or one payment platform can improve at recognizing fraud that looks like what it saw in training, while getting worse, in relative terms, at recognizing new fraud patterns that were not well represented in that data, because the model's decision boundary has shifted to fit the majority pattern it was shown most often. Adding a large volume of data from that same narrow distribution does not fix this; it can make the model more confidently specialized to exactly the cases it already handled well, at the expense of the cases it was already weaker on, which is precisely the wrong direction if the goal is broad, reliable performance.

When more data helps vs when it can hurt

The question that matters is not "how much" but "what kind"

Practitioners who have been burned by this once tend to stop treating dataset size as the primary lever and start asking more specific questions before adding a large new batch of data: where did this data come from, and could that source introduce a systematic pattern that is not the pattern we want; is the labeling process for this new data as reliable as the process used for the original data; does this new data represent the same population and conditions the model will face in real use, or a narrower slice of it. None of this means more data is a bad thing in general, in the majority of practical situations, a reasonably sized, well-collected increase in data does help. The point is narrower and more specific: "more data" is not, by itself, a fix. It is an amplifier, and it amplifies whatever pattern, genuine or spurious, is present in what gets added.

Synthetic and augmented data carry the same rule, just less obviously

The temptation to solve a data-scarcity problem by generating synthetic examples, or by augmenting existing data through transformations, follows the same underlying rule as any other data addition: it helps only to the extent that it introduces genuine, relevant variation, and it can hurt if it introduces an artificial pattern the model learns as if it were real. Image augmentation that rotates or crops photos generally introduces harmless, realistic variation, because a photo of the same object still looks like a real photo after those changes. A synthetic data generation process that inadvertently produces examples with a subtle, consistent artifact, a particular rendering quirk, an unrealistic combination of features that would never occur together in real data, can teach a model to rely on that artifact as if it were a genuine signal, which then fails exactly the way any other biased-data problem fails once the model meets real, unaugmented data.

A held-out test set only protects you if it stays untouched

Practitioners are taught early to keep a portion of data aside as a test set, never used during training, specifically to get an honest read on how the model performs on data it has not seen. This protection quietly erodes in a common and easy-to-miss way: if a team repeatedly checks performance on that same held-out set while iterating on the model, adjusting features, tuning hyperparameters or trying different architectures based on how each attempt scores against it, the test set gradually stops functioning as a unseen benchmark: the choices being made are, indirectly, being optimized against it, even though no single training run technically included it. A model that looks like it is improving under this process can just be increasingly well-fit to the specific quirks of that one test set, and the "more data helped" conclusion drawn from it may not hold up against data the test set never touched either.

The fix is rarely "collect even more": it is usually "look harder at what you have"

When a model underperforms, the instinctive response is often to request more data, since that is the lever that feels most directly within reach and most obviously correlated with model quality in the general case. Experienced teams frequently find that the more productive next step is closer examination of the existing dataset: reading a sample of examples by hand, checking for label inconsistency between similar cases, looking for an entire category of input the data barely represents, or checking whether the feature the model is relying on most heavily makes sense as a genuine predictor rather than a coincidental one. This kind of manual, close-reading audit does not scale the way collecting more data does, which is part of why it gets skipped under time pressure, but it is often the step that explains why the model is underperforming, in a way that adding volume to an already-flawed dataset cannot fix.

Before assuming more data will improve a model

• Check performance on a held-out, representative test set, not just training accuracy, to see if the model is improving or just memorizing.
• Investigate how the new data was collected and labeled, and whether that process could introduce a systematic bias.
• Compare the distribution of the new data against the real conditions the model will be used in.
• Audit a sample of the new data manually for label quality before assuming volume implies value.
• Be specific about what pattern you want the model to learn, so you can check whether the new data contains it.

All Ground Truth articles · Career Insights · Discover jobs · Companies