Data Cleaning Projects for Beginners

A good beginner data cleaning project is one that answers one clear question from one public dataset, applies a short list of written rules, keeps rejected rows in an exceptions file, and proves with a count that nothing went missing. The six projects below follow that shape. Each names a free dataset with a checked license, the question to answer, the skills it practices and what to hand in at the end.

The common mistake with first projects is scope. "Clean this whole dataset" never ends. "Find monthly revenue net of cancellations" does.

Key Takeaways

• Scope every project to one question. The question decides which columns need cleaning and which can be ignored.
• Write your cleaning rules down before you write code, and keep every rejected row with a reason.
• Finish with a reconciliation: rows in must equal clean rows plus exceptions.
• Hand in a short README with the question, source and license, rules, exceptions and results. That's what makes a project readable to someone else.
• Start with the smaller projects (1 and 2) and move to joins and reports once those feel routine.

What makes a good beginner project?

A beginner project should be small enough to finish in a few sessions and real enough to involve decisions. In practice that means:

• One dataset, or two that join. More sources multiply the problems before you've solved the first ones.
• One question. It tells you when you're done.
• A known license. You'll want to publish the result, so check that the data allows it.
• A visible quirk to handle. Cancellations, duplicates, missing values written several ways. Without one, there's nothing to clean.
• A deliverable. A cleaned file, an exceptions file and a README, not just a notebook that ran once.

If you'd rather start with a fully specified brief, the projects on this site come with a scenario, data and the checks your output must pass. The six below are open-ended: you write the rules.

The six projects at a glance

Project 1: Monthly revenue net of cancellations

Dataset: UCI Online Retail, 541,909 transaction lines from a UK online retailer, licensed CC BY 4.0. The UCI page notes that an invoice number starting with "c" marks a cancellation.

Question: What was revenue per month, and how much of it was reversed by cancellations?

What you'll practice: reading ids and codes as text, converting prices to exact decimals instead of floats, separating cancellations from sales, and checking for repeated lines before you sum anything.

The real file is an Excel workbook with columns such as InvoiceNo, Quantity, UnitPrice and InvoiceDate. Read it with every column as text, rename the columns, and parse InvoiceDate explicitly before you take the month. The core logic then looks like this in pandas, shown on a simplified six-line CSV with ISO dates (the values are illustrative):

from decimal import Decimal, InvalidOperation

import pandas as pd

raw = pd.read_csv("lines.csv", dtype=str, keep_default_na=False)
repeats = raw[raw.duplicated(keep=False)]

def to_decimal(text):
    try:
        return Decimal(text.strip())
    except InvalidOperation:
        return None

lines = raw.assign(
    qty=pd.to_numeric(raw["quantity"].str.strip(), errors="coerce"),
    price=raw["unit_price"].map(to_decimal),
    is_cancellation=raw["invoice"].str.upper().str.startswith("C"),
    month=raw["invoice_date"].str[:7],
)

bad = lines["qty"].isna() | lines["price"].isna()
exceptions = lines[bad].assign(reason="quantity or price not a number")
clean = lines[~bad].copy()
clean["amount"] = [Decimal(int(q)) * p for q, p in zip(clean["qty"], clean["price"])]

print(clean.groupby(["month", "is_cancellation"])["amount"].sum())
print(len(repeats), "repeated lines to review")
print(len(raw), "rows in =", len(clean), "clean +", len(exceptions), "exceptions")

Output with pandas 3.0:

month    is_cancellation
2010-12  False               26.40
         True               -27.50
2011-01  False               15.00
         True                -9.90
Name: amount, dtype: object
0 repeated lines to review
6 rows in = 5 clean + 1 exceptions

One line had abc as its quantity, so it went to exceptions with a reason instead of disappearing. The sample has no exact repeats, but on the real file the repeats table is where you'd look before deciding whether a repeated line is a duplicate export or a genuine second purchase. Amounts are Python Decimal values, so 26.40 is exact rather than a float that prints close to it. Python's decimal documentation explains why that matters for money, and the money and exact decimals guide covers the SQL side.

Deliverable: a monthly table with sales, cancellations and net revenue; an exceptions file; and a line in your README stating rows in, clean rows and exceptions. If you want to rehearse the moves first, the free order-line export exercise uses the same kind of data at a smaller scale.

Project 2: Turn a messy spreadsheet into a tidy table

Dataset: the Portal Project Teaching Database on figshare, licensed CC0. It includes survey_data_spreadsheet_messy.xls and untidy-portal-data.xlsx next to clean CSV versions. Data Carpentry's spreadsheet lesson describes its teaching subset as "intentionally 'messed up'".

Question: Can you combine the messy spreadsheet's survey tabs into one tidy table in the same structure as the database's clean CSV files?

What you'll practice: turning a spreadsheet laid out for people into one row per observation, making codes and dates consistent, and deciding what to do with anything that isn't data, such as notes or formatting. Profile the messy file first and list what you find before you fix anything.

Why it suits beginners: the lesson sets up a concrete scenario. Two field assistants each recorded a year of surveys (2013 and 2014) in their own way, and your job is to combine the two tabs into one table without modifying the original file. The database's clean CSVs give you a target structure to aim for: one row per observation, consistent column names and codes. They aren't a row-by-row answer key for this file, so judge your result by whether every observation in the tabs ends up in exactly one row of your table. A single spreadsheet row in a messy layout can hold more than one observation, so count observations, not spreadsheet rows. The lesson's instructor notes also flag dates typed without a year, which the spreadsheet filled in with the wrong one.

Deliverable: your tidy CSV, the script or steps that produce it, an observation count per tab and in the combined table, and a list of the problems you found with the rule you applied to each.

Project 3: Clean and deduplicate a contact list

Dataset: build your own. Write 30 to 50 fake contact rows in a spreadsheet, and plant the problems on purpose: padded names, mixed-case emails, the same person entered twice with different spellings, a blank email next to a missing one. Making your own data means there's no license question and you know the right answer.

Question: How many distinct people are on the list, and which row should represent each one?

What you'll practice: trimming and case rules, a written email policy (what you lowercase, what you keep), matching on evidence rather than guesses, and a survivor rule that picks one row per person deterministically.

The interesting decisions are about what counts as the same person. Two rows with the same email are strong evidence; two rows with the same name are not. The deduplication and entity resolution guide walks through those trade-offs, and the free normalize emails under a declared policy exercise practices the email rules.

Deliverable: the deduplicated list, a review file of pairs you weren't sure about, and the rules you applied.

Project 4: Complaint trends with a caveat note

Dataset: the CFPB's Consumer Complaint Database, downloadable as a zipped CSV. The CFPB describes the data as "freely available for anyone to use, analyze, and build on", and states that it is not a statistical sample of consumers' experiences.

Question: How did complaint volume for one product change by month over a recent year?

What you'll practice: filtering to a scope and saying so, parsing dates explicitly, checking category labels for near-duplicates, and writing down what the numbers can't support.

The caveat is the point of this project. Because complaints aren't a statistical sample, a rise in complaints doesn't by itself show that a product got worse. Writing that limitation clearly is a real analyst skill, and the guide to writing a data quality report gives a structure for it.

Deliverable: a monthly count table, a short chart if you like, and a paragraph of limitations.

Project 5: Join listings to reviews without fan-out

Dataset: Inside Airbnb for one city, licensed CC BY 4.0. Use the detailed listings file and the reviews file.

Question: What is the review count per listing, and which listings have no reviews at all?

What you'll practice: stating the row grain of each file before joining, checking that the listing identifier is unique on the listings side, counting rows before and after the join, and reporting reviews whose listing identifier doesn't appear in the listings file instead of silently dropping them.

Joins are where row counts quietly change. A join on a key that isn't unique multiplies rows, and an inner join drops rows that have no match. The guide to checking joins before trusting them has the checks to run.

Deliverable: a per-listing table with review counts (including zeros), an orphans file, and before-and-after row counts.

Project 6: A data quality report on a museum collection

Dataset: the MoMA collection files or the Met's open access CSV, both released under CC0. Both museums say their records are incomplete or not fully reviewed.

Question: How complete and consistent is this collection data, column by column?

What you'll practice: profiling at scale, measuring completeness, spotting columns that mix formats, and turning findings into a report someone could act on, without changing the data.

This project is different: you don't clean anything. You describe what you'd need to clean and why. That's the first step of every real cleaning job, and it's good practice in separating what you observe from what you'd decide.

Deliverable: a one- or two-page report with a table of columns, completeness, issues found and a recommended rule for each.

How to present a beginner project

Whatever you build, the README matters as much as the code. Keep it short and include:

1. The question in one sentence.
2. Source and license, with a link and the download date.
3. Rules, one line each, in the order they run.
4. Exceptions: how many rows, and the reasons.
5. Reconciliation: rows in, clean rows, exception rows.
6. Results and limitations: the answer, plus what the data can't tell you.

The portfolio project guide shows how to turn this into something a hiring manager can skim in two minutes. For more datasets with checked licenses, see free messy datasets for practice; if you want structured practice before starting, the 30-day data cleaning practice plan builds up to a project like these.

Frequently asked questions

Which project should I start with?

Project 1 or 2. Both are small, have a clear quirk to handle, and give you a way to check your work: a reconciliation for project 1, and for project 2 an observation count per tab plus the clean CSVs as a target structure.

SQL or pandas for these projects?

Either works for all six. Projects 1, 4 and 5 suit DuckDB well because the files are large and the work is mostly filtering, grouping and joining. Projects 2 and 3 are comfortable in pandas. Doing one project in both is good practice in its own right.

Do I need to clean every column?

No. Clean the columns your question needs and leave the rest as they are. Saying "out of scope" in the README is a sign of judgment, not laziness.

Related reading

• Data Cleaning Skills on a Resume: What to Write and How to Prove It
• Cleaning E-commerce Order Data: A Worked Example

Sources

All dataset pages retrieved November 5, 2026.

• UCI Machine Learning Repository, Online Retail (CC BY 4.0).
• Morgan Ernest, James Brown, Thomas Valone, Ethan P. White, Portal Project Teaching Database, figshare (CC0).
• Data Carpentry, Data Organization in Spreadsheets for Ecologists, including the formatting data episode that describes the two-tab messy file.
• Consumer Financial Protection Bureau, Consumer Complaint Database.
• Inside Airbnb, Get the data (CC BY 4.0).
• The Museum of Modern Art, collection repository (CC0).
• The Metropolitan Museum of Art, Open Access repository (CC0).
• Python, decimal module documentation.
• The pandas example was run with pandas 3.0.2 on a six-line illustrative sample.