Data Frog Field notes in data and Python

Data Wrangling & Analysis

Make messy real-world data analyzable, repeatably. This track covers pandas in depth, data cleaning, missing values, joins and reshaping, feature engineering, and exploratory analysis — with NumPy as the substrate.

Start with the library landscape, then work the NumPy series.

Pandas, cleaning, joins, reshaping, and EDA lessons are planned next. See the curriculum dashboard for the roadmap.

In review — 1 draft awaiting publication

Planned lessons (29)

Wave 1 — first vertical slice (production-ready) — 29 lessons

pandas-core

  • Reading Files concept · beginner · M CSV, Excel, and Parquet in, and the ways each silently mangles your data.
  • First Look concept · beginner · S head, describe, value_counts: a disciplined first pass over new data.
  • Selecting and Filtering concept · beginner · M loc, iloc, boolean masks, and query: get exactly the rows you mean.

transforming

  • Missing Data concept · intermediate · M The three kinds of missingness and what each licenses you to do.
  • Cleaning Strings and Dates concept · intermediate · L Tame the mess: whitespace, formats, parse errors, duplicates.
  • dtype Hygiene concept · intermediate · M Downcasting, nullable types, and the memory you didn't know you were wasting.
  • GroupBy: Split, Apply, Combine concept · intermediate · M The single most useful move in pandas, from counting to transforming.
  • Joins and merge concept · intermediate · M Merge mechanics: key shapes, validation, and the rows that vanish.
  • Reshaping and Tidy Data concept · intermediate · M melt, pivot, and the tidy-data discipline behind both.
  • Time Series Essentials concept · intermediate · M DatetimeIndex, resampling, and timezones without fear.
  • Rolling Windows concept · intermediate · S Rolling statistics, expanding windows, and what they smooth away.
  • Text in pandas concept · intermediate · S The .str accessor, vectorized regex, and when to give up and loop.
  • Categoricals concept · intermediate · S Ordered categories, memory savings, and sorting that finally makes sense.

performance

  • Vectorization concept · intermediate · M Replace loops with array operations, and know when you can't.
  • pandas Performance concept · advanced · M Profiling pandas: where the time goes and which idiom to switch to.

eda

  • A Repeatable EDA Pass example · intermediate · L A structured first-pass EDA on a real messy dataset.
  • Finding Surprises example · intermediate · M Distributions, outliers, and leaks: catch them before the model does.

features

  • Feature Engineering: Numeric concept · intermediate · M Scaling, transforms, binning, and interaction features that mean something.
  • Feature Engineering: Categorical concept · intermediate · M Encoding choices: one-hot, ordinal, target, and their leakage risks.
  • Feature Engineering: Dates and Text concept · intermediate · M Calendar features, lags, and the text signals worth extracting.

quality

  • Outliers: Detect, Understand, Decide concept · intermediate · M Find outliers, understand them, and decide with a reason.
  • Data Validation concept · intermediate · S Assert your assumptions about shape, ranges, and uniqueness.
  • Data Dictionaries reference · beginner · S Documenting a dataset so future-you survives.
  • Parquet and Columnar Formats concept · intermediate · S Why analytics loves columnar storage.

sources

  • APIs to Tables concept · intermediate · M requests, pagination, and JSON chaos turned into a tidy DataFrame.

sql

  • SQL for Analysis concept · intermediate · L SELECT through window functions: the SQL an analyst actually needs.
  • SQL vs pandas concept · intermediate · M Where the work should run, and how to stop pretending it's religious.

projects

  • Case Study: A Messy Real Dataset project · intermediate · L From an ugly raw file to an analysis-ready table, documented end to end.
  • Case Study: Three Tables, One Honest Table project · intermediate · L Multiple sources merged with validated keys and a real data dictionary.