Do Data Scientists Really Spend 80% of Their Time Cleaning Data?

No, not according to the surveys usually cited for it. The "80%" figure traces back to a 2014 New York Times article that gave a range of 50 to 80 percent based on interviews and expert estimates, and to a misreading of a 2016 CrowdFlower survey that measured something else. Later self-reported surveys put data preparation at a large but smaller share. In those surveys, data preparation means loading and cleaning data before any modelling. Anaconda's respondents reported about 45% of their time in 2020, and about 38% in 2022 according to I Programmer's report on that survey.

That is still a lot: in Anaconda's 2020 figures, data preparation (45%) outweighed model selection, training and deployment combined (about 34%). The number is wrong; the underlying point, that cleaning is a large part of the job, holds up.

Key Takeaways

• The 80% figure is the top of a 50–80% range from a 2014 New York Times article, which was based on interviews and expert estimates rather than a survey.
• CrowdFlower's 2016 survey found that 60% of respondents named cleaning and organizing as the task they spend the most time on. That is a share of people, not a share of hours, and adding the 19% who named collecting data gives the familiar "about 80%".
• Surveys that asked about shares of time report less: about 45% for data preparation in Anaconda's 2020 survey, and about 38% in its 2022 survey as reported by I Programmer.
• All of these are self-reported estimates with category boundaries that differ from survey to survey, so treat any single percentage as approximate.
• The practical conclusion doesn't depend on the exact number: cleaning is a large, skilled part of data work, and it rewards practice.

Where does the 80% claim come from?

The claim has two main roots, and neither is a measurement of 80% of working time.

The first is an August 2014 New York Times article by Steve Lohr on data "janitor work". Leigh Dodds traced the claim in detail in January 2020 (Leigh Dodds). In his summary, the article said data scientists spend 50 to 80 percent of their time collecting and preparing data, based on interviews and expert estimates rather than a survey. CrowdFlower's own 2016 report referred back to the same article, saying the Times had "estimated that up to 80%" of a data scientist's time goes to this kind of work (CrowdFlower 2016 Data Science Report, p. 7). "Up to 80%" is the top of a range, not a typical value.

The second root is the CrowdFlower survey itself, which is where most later articles and slide decks say the number comes from.

What did the CrowdFlower survey actually measure?

CrowdFlower's 2016 report asked data scientists what they spend the most time doing. It then charted the share of respondents who picked each task (CrowdFlower 2016 Data Science Report, p. 6):

The report's own summary is careful: "3 out of every 5 data scientists we surveyed actually spend the most time cleaning and organizing data." That is a statement about how many people put cleaning first, not about how much of their week it fills. Someone who spends 35% of their time cleaning and less on everything else would still answer "cleaning".

Add the 60% who chose cleaning to the 19% who chose collecting data, and you get 79%. Dodds points out that this sum is the only way to reach roughly 80% from the CrowdFlower data, and that the claim usually reaches readers through a chain of secondary citations: in his tracing, an IBM marketing page cited a 2018 Harvard Business Review post, which cited a 2017 IBM blog, which cited the 2016 CrowdFlower survey (Leigh Dodds).

Two further caveats apply. The report doesn't state how many people responded or how they were selected; it says only that respondents ranged from high-level chief data officers to people just starting in the field, with the largest group (43%) having two to five years of experience. And the same survey found cleaning was also the least enjoyed task: 57% named cleaning and organizing data as the least enjoyable part of the job (p. 7). The "80%" claim often travels with that complaint attached, which helps explain why it caught on.

What do surveys that measured time report?

Surveys that asked people to estimate shares of their working time give lower numbers, though still large ones.

For 2020, InfoWorld reported Anaconda's results in June 2020 (InfoWorld). Respondents estimated 19% of their time on data loading and 26% on data cleaning, against roughly 34% in total for model selection, training and scoring, and deployment. For 2022, Anaconda's survey drew 3,493 respondents from 133 countries and regions (Anaconda), and I Programmer's write-up reports that "around 38% of time is spent on data preparation and cleansing" (I Programmer, November 2022).

Dodds also collected figures from other surveys. Kaggle's 2018 survey put gathering data at about 11% and cleaning at about 15% of time, about 26% combined. Figure Eight's 2019 report found that 73.5% of technical respondents spent 25% or more of their time managing, cleaning or labeling data; like CrowdFlower's figure, that is a share of people, not a share of hours, so it can't be compared directly with the time-share numbers.

Why the exact number is hard to pin down

Even a well-run survey can't produce one true percentage, for three reasons.

• Categories differ. One survey's "cleaning" is another's "loading", "labeling" or "organizing". Collecting data, joining it, checking it and fixing it blur into each other in real work, and every survey splits them differently.
• The numbers are self-reported. People estimating their own week may overweight the tasks they find most frustrating. None of these surveys timed anyone's work.
• Samples differ. Anaconda's respondents include students, academics and professionals, not only data scientists, and CrowdFlower didn't publish its sample size. Results from different populations aren't directly comparable.

So the honest summary is a range: in the surveys that measured shares of time (Kaggle 2018 at about 26%, Anaconda at 38–45%), data preparation is reported as somewhere between a quarter and a half of the job, and in Anaconda's 2020 survey it outweighed all the modeling steps combined.

Does it matter whether it's 40% or 80%?

For a hiring manager or a student deciding what to learn, the exact figure matters less than what it points to. A large share of data work goes into getting data into a state where results can be trusted, and that work involves real decisions: what counts as missing, which duplicate survives, what to do with values that don't fit a rule. Those decisions change the final numbers.

Dodds makes a related argument: calling it "janitor work" undervalues the part of the job where people learn what their data actually contains. Time spent profiling a new dataset is often where analysts find the problems that would otherwise have reached a report.

The time also isn't fixed. Much of it goes into the same checks repeated by hand. Writing validation rules with exception reports and keeping steps in a reproducible cleaning pipeline turns one-off fixes into checks that run every time, which is how teams shrink the share without skipping the work. A clear definition of data cleaning helps here too, because it separates the cleaning itself from collecting and modeling.

What this means if you're learning data cleaning

If cleaning is a quarter to a half of the job, it's worth being good at, and it's a skill that improves with deliberate practice rather than with reading alone. A few practical steps:

• Practice the core moves on small, checked tasks. The free order-line export exercise and validating ranges with ordered reasons cover the decisions that come up most.
• Practice explaining trade-offs. Interviews for data roles can probe how you'd handle bad data, and the data quality trade-off questions are good prompts.
• Build a routine. How to practice data cleaning covers the habits, and the 30-day data cleaning practice plan lays them out day by day.

Frequently asked questions

Is it true that data scientists spend 80% of their time cleaning data?

Not according to the surveys that measured time. The 80% figure is the top of a 2014 estimate range and a misreading of a 2016 survey. Anaconda's time-share surveys report about 45% (2020) and, as reported by I Programmer, about 38% (2022) for data preparation, which is still a large share of the job.

Where did the 80/20 data cleaning rule come from?

From a mix of the New York Times' 2014 "50 to 80 percent" estimate and CrowdFlower's 2016 survey, in which 60% of respondents said cleaning took the most time and 19% said collecting data did. Adding those two shares of respondents gives about 80%, which is how the figure is often repeated.

Has the share of time spent on data preparation gone down?

Anaconda's figures went from about 45% in 2020 to about 38% in 2022 (the 2022 figure as reported by I Programmer). Because both are self-reported and the questions and samples changed between years, that is a suggestive difference rather than proof of a trend.

Related reading

• What Is Data Cleaning? A Plain-English Guide With Examples
• Data Wrangling, Data Cleaning and Data Munging: What's the Difference?
• Data Cleaning for Machine Learning: What Changes and What Doesn't

Sources

All pages retrieved November 5, 2026.

• CrowdFlower, 2016 Data Science Report (PDF copy hosted at the University of Houston), pp. 4–7.
• Leigh Dodds, "Do data scientists spend 80% of their time cleaning data? Turns out, no?", January 31, 2020 (updated September 29, 2020). Source for the New York Times 2014 summary, the citation chain, and the Kaggle 2018 and Figure Eight 2019 figures.
• Serdar Yegulalp, "For data scientists, drudgery is still job #1", InfoWorld, June 30, 2020 (Anaconda 2020 State of Data Science).
• Anaconda, 2022 State of Data Science report page.
• Sue Gee, "Data On Data Scientists", I Programmer, November 2, 2022 (Anaconda 2022 State of Data Science).
• Steve Lohr, "For Big-Data Scientists, 'Janitor Work' Is Key Hurdle to Insights", The New York Times, August 2014 (cited via Dodds and the CrowdFlower report; not retrieved directly).