EDA & Feature Engineering

Models are only as good as the data you feed them, and most real-world accuracy gains come from understanding and reshaping that data — not from fancier algorithms. This lesson covers the two crafts that dominate working data science: exploratory data analysis (EDA), where you interrogate the data before modeling, and feature engineering, where you turn raw columns into signals a model can actually use.

What EDA is looking for

EDA isn't aimless plotting — it's a checklist of questions, each of which changes what you do next:

• What's the target, and is it balanced? A 95/5 class split changes your metrics and your baseline (next lesson digs into this).
• What type is each feature? Numeric, categorical, ordinal, datetime, free text — each needs different preprocessing.
• How much is missing, and where? A column that's 80% empty is probably a drop; a column missing 2% is an impute.
• What do the distributions look like? Skewed features (like incomes or fares) may benefit from a log transform or binning.
• Any outliers? Are they data-entry errors or real (and important) rare events?
• Which features relate to the target? Group means, correlations, and simple plots reveal which columns carry signal.

Let's run a compact EDA on a synthetic passenger dataset — a small ship manifest with survival labels, in the spirit of the famous Titanic problem:

Python — runs in your browser

Run

import numpy as np import pandas as pd rng = np.random.default_rng(42) n = 500 pclass = rng.choice([1, 2, 3], n, p=[0.25, 0.20, 0.55]) sex = rng.choice(["male", "female"], n, p=[0.65, 0.35]) age = rng.normal(30, 13, n).clip(1, 80).round(0) age[rng.random(n) < 0.20] = np.nan # 20% of ages unknown fare = (rng.lognormal(2.2, 0.8, n) * (4 - pclass)).round(2) family = rng.poisson(0.9, n) p_survive = 0.12 + 0.55 * (sex == "female") + 0.15 * (pclass == 1) survived = (rng.random(n) < p_survive).astype(int) df = pd.DataFrame({"pclass": pclass, "sex": sex, "age": age, "fare": fare, "family": family, "survived": survived}) print("--- target balance ---") print(df["survived"].value_counts(normalize=True).round(3)) print("\n--- missing values (fraction per column) ---") print(df.isna().mean().round(3)) print("\n--- numeric distributions ---") print(df[["age", "fare", "family"]].describe().round(1)) print("\n--- survival rate by group ---") print(df.groupby("sex")["survived"].mean().round(3)) print(df.groupby("pclass")["survived"].mean().round(3))

Five minutes of EDA already wrote our modeling plan: the target is imbalanced (roughly one third survived), age needs imputation, fare is heavily right-skewed (compare its mean to its median), and sex and pclass clearly carry signal. On larger datasets you'd add histograms (df["fare"].hist()), correlation matrices (df.corr(numeric_only=True)), and count plots per category — same questions, more pictures.

Handling missing values

Two families of fixes:

• Drop. Remove rows (df.dropna()) when very few are affected, or drop a whole column when most of it is missing — there's little left to learn from. Always drop rows whose target is missing.
• Impute. Fill numeric gaps with the median (robust to outliers) or mean; fill categorical gaps with the most frequent value. In scikit-learn that's SimpleImputer, which you met inside pipelines — and the pipeline is exactly where imputation belongs, so its statistics come from training folds only.

A third, underrated option: add a boolean age_missing indicator column. Sometimes the fact that a value is missing is itself predictive.

Encoding categorical features

Models compute with numbers, so categories must be encoded:

• One-hot encoding (OneHotEncoder, or pd.get_dummies for quick exploration) creates one 0/1 column per category. Right choice for nominal categories with no order: port, city, color.
• Ordinal encoding (OrdinalEncoder) maps categories to integers. Only right when the order is real — small, medium, large — because models will treat the numbers as ordered and evenly spaced.

Beware one-hot exploding on high-cardinality columns (thousands of zip codes); grouping rare categories into an "other" bucket is a simple, effective fix.

Creating new features

This is where domain thinking beats algorithms. Classic moves:

• Boolean flags — is_alone from a family count.
• Ratios and combinations — fare per family member; total family size from siblings plus parents.
• Binning — turn numeric age into categories like child / teen / adult, which can capture non-linear effects and tame outliers.
• Datetime parts — from a timestamp, extract hour, weekday, month; "purchases spike on weekends" is invisible to a raw timestamp.
• Text extraction — pull a title like "Mr" or "Dr" out of a name string.

Python — runs in your browser

Run

import numpy as np import pandas as pd rng = np.random.default_rng(42) n = 500 pclass = rng.choice([1, 2, 3], n, p=[0.25, 0.20, 0.55]) sex = rng.choice(["male", "female"], n, p=[0.65, 0.35]) age = rng.normal(30, 13, n).clip(1, 80).round(0) fare = (rng.lognormal(2.2, 0.8, n) * (4 - pclass)).round(2) family = rng.poisson(0.9, n) p_survive = 0.12 + 0.55 * (sex == "female") + 0.15 * (pclass == 1) survived = (rng.random(n) < p_survive).astype(int) df = pd.DataFrame({"pclass": pclass, "sex": sex, "age": age, "fare": fare, "family": family, "survived": survived}) # 1. Boolean flag df["is_alone"] = (df["family"] == 0).astype(int) # 2. Ratio feature df["fare_per_person"] = (df["fare"] / (df["family"] + 1)).round(2) # 3. Binning a numeric column df["age_group"] = pd.cut(df["age"], bins=[0, 12, 18, 40, 80], labels=["child", "teen", "adult", "senior"]) # 4. Datetime parts (synthetic boarding timestamps) boarding = pd.to_datetime("2026-04-01") + pd.to_timedelta( rng.integers(0, 72, n), unit="h") df["boarding_hour"] = boarding.hour df["boarding_day"] = boarding.day_name() print(df[["fare", "family", "is_alone", "fare_per_person", "age_group", "boarding_hour", "boarding_day"]].head()) print("\nsurvival rate by age group:") print(df.groupby("age_group", observed=True)["survived"].mean().round(3))

After creating features, always sanity-check them against the target the way we did with groupby — a new feature that doesn't separate the target at all is probably not pulling its weight.

Scaling, one more time

Recap from the KNN lesson: distance- and gradient-based models (KNN, SVMs, linear models with regularization, neural networks) need features on comparable scales — StandardScaler or MinMaxScaler inside your pipeline. Tree-based models split one feature at a time and don't care about scale. When in doubt, scale: it never hurts, and forgetting it can quietly cripple half your model zoo.

Check your understanding

4 questions · free

1. Q1.A column is missing 85% of its values. What's usually the best first move?

   AImpute it with the meanBDrop the column — there's too little information leftCDrop every row with a missing valueDFill it with zeros

2. Q2.Why is one-hot encoding preferred over ordinal encoding for a "port of embarkation" column?

   AOne-hot is faster to computeBPorts have no natural order, and ordinal integers would invent oneCOrdinal encoding can't handle stringsDOne-hot uses less memory

3. Q3.The mean fare is 45 but the median is 18. What does this tell you?

   AThe data contains missing valuesBThe distribution is right-skewed — a few very large fares pull the mean upCThe fares are normally distributedDThe median was computed incorrectly

4. Q4.Which of these is an example of data leakage during feature engineering?

   ACreating an is_alone flag from the family columnBComputing bin edges for fare from the full dataset (train + test), then binning bothCExtracting the weekday from a timestampDOne-hot encoding inside a pipeline

Exercise: Engineer and evaluate two new features

Regenerate the synthetic passenger dataset and engineer two new features: is_child (1 when age is below 12) and family_bucket (bin the family count into alone, small for 1–2, and large for 3+ using pd.cut). Print the survival rate for each group of both features, plus the group sizes. Do the new features separate the target — and are any groups too small to trust?

Next up: before celebrating any model's score, you need something to compare it against — baselines and benchmarks.