scikit-learn Pipelines & Workflow

You now know the workflow (split → fit → predict → evaluate) and you've seen that preprocessing like scaling must be fit on training data only. Doing that by hand gets error-prone fast — real datasets need imputation and scaling and encoding, each with its own fit-on-train rule. scikit-learn's answer is the Pipeline: bundle every step into a single object that behaves like one model. This lesson is the glue that holds the rest of the course together.

One interface: fit, predict, transform

Everything in scikit-learn is an estimator sharing a tiny, consistent API:

• est.fit(X_train, y_train) — learn from data. Models learn parameters; preprocessors learn statistics (a scaler learns means, an encoder learns categories).
• est.predict(X) — for models (predictors): output labels or numbers.
• est.transform(X) — for transformers (preprocessors): output a modified copy of the data.
• est.fit_transform(X) — fit then transform in one call (train data only!).

Because every scaler, encoder, imputer, and model speaks this same language, you can swap a KNN for a random forest — or a MinMax scaler for a standard one — without changing the surrounding code. That uniformity is what makes pipelines possible.

Leakage: the silent score inflator

Data leakage is when information from the test set sneaks into training. The model's scores look great, then reality disappoints. The most common leaks are mundane:

• Scaling with statistics from all rows — the test set's mean has leaked into training.
• Imputing missing values using the full dataset — same problem.
• Tuning hyperparameters against the test set — after enough peeks, the test set is effectively memorized.

Pipeline: preprocessing and model as one estimator

A Pipeline chains transformers and ends with a model. When you call fit, it fit-transforms each step on training data in sequence; when you call predict or score, it only transforms — the fit-on-train rule is enforced automatically:

Python — runs in your browser

Run

from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split, cross_val_score from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler from sklearn.neighbors import KNeighborsClassifier X, y = load_iris(return_X_y=True) X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.3, stratify=y, random_state=42 ) pipe = Pipeline([ ("scaler", StandardScaler()), ("knn", KNeighborsClassifier(n_neighbors=5)), ]) pipe.fit(X_train, y_train) # scaler fit on train, then knn fit print(f"test accuracy: {pipe.score(X_test, y_test):.3f}") # Cross-validation: 5 different train/validation splits, scaler refit each time scores = cross_val_score(pipe, X_train, y_train, cv=5) print(f"5-fold CV : {scores.round(3)}") print(f"mean ± std : {scores.mean():.3f} ± {scores.std():.3f}")

The cross_val_score call is doing something subtle and important. A single train/test split can be lucky or unlucky; k-fold cross-validation splits the training data into k parts, trains on k−1 and validates on the remaining one, rotating k times — like judging a student on both a midterm and a final instead of one exam. The mean of the k scores is a far more robust estimate. And because we passed the pipeline, the scaler is refit inside every fold — zero leakage, zero manual bookkeeping.

Mixed columns: ColumnTransformer

Real tables mix numeric columns (scale them, impute with the median) and categorical columns (impute with the most frequent value, one-hot encode them). ColumnTransformer routes each group of columns through its own mini-pipeline:

Python — runs in your browser

Run

import numpy as np import pandas as pd from sklearn.compose import ColumnTransformer from sklearn.impute import SimpleImputer from sklearn.pipeline import Pipeline from sklearn.preprocessing import OneHotEncoder, StandardScaler rng = np.random.default_rng(42) n = 8 df = pd.DataFrame({ "age": [22, 35, np.nan, 58, 41, np.nan, 29, 63], "fare": [7.9, 71.3, 8.1, 26.6, np.nan, 13.0, 30.1, 77.9], "sex": ["male", "female", "female", "male", "male", None, "female", "male"], "port": ["S", "C", "S", "S", "Q", "S", "C", "S"], }) print(df) numeric_pipe = Pipeline([ ("imputer", SimpleImputer(strategy="median")), ("scaler", StandardScaler()), ]) categorical_pipe = Pipeline([ ("imputer", SimpleImputer(strategy="most_frequent")), ("onehot", OneHotEncoder(handle_unknown="ignore")), ]) preprocessor = ColumnTransformer([ ("num", numeric_pipe, ["age", "fare"]), ("cat", categorical_pipe, ["sex", "port"]), ]) X_ready = preprocessor.fit_transform(df) print("\nshape after preprocessing:", X_ready.shape) print(preprocessor.get_feature_names_out())

Four messy columns in; seven clean numeric columns out — missing values filled, numerics standardized, categories expanded into one-hot indicator columns. handle_unknown="ignore" keeps predictions from crashing if a category appears at test time that training never saw.

Tuning the whole pipeline: GridSearchCV

Hyperparameters (like KNN's n_neighbors) shouldn't be tuned against the test set. GridSearchCV tries every parameter combination using cross-validation inside the training data, then refits the best one. Name pipeline steps and address their parameters with a double underscore, as in knn__n_neighbors:

Python — runs in your browser

Run

from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split, GridSearchCV from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler from sklearn.neighbors import KNeighborsClassifier X, y = load_iris(return_X_y=True) X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.3, stratify=y, random_state=42 ) pipe = Pipeline([ ("scaler", StandardScaler()), ("knn", KNeighborsClassifier()), ]) param_grid = { "knn__n_neighbors": [3, 5, 9, 15], "knn__weights": ["uniform", "distance"], } search = GridSearchCV(pipe, param_grid, cv=3) search.fit(X_train, y_train) print("best params :", search.best_params_) print(f"best CV score: {search.best_score_:.3f}") print(f"test accuracy: {search.score(X_test, y_test):.3f}")

That's 4 × 2 = 8 combinations, each cross-validated 3 times — 24 fits, all leak-free because scaling happens inside each fold. On real projects the grid is bigger and you'd add n_jobs=-1 to parallelize across CPU cores (and consider RandomizedSearchCV when the grid explodes).

The complete modern workflow, then, in one breath: split → build preprocessor + model into a pipeline → grid-search with CV on the training set → evaluate once on the test set. This skeleton carries you through nearly every tabular ML problem.

Check your understanding

5 questions · free

1. Q1.Which method does a transformer (like StandardScaler) use to modify data after fitting?

   ApredictBtransformCscoreDapply

2. Q2.You call scaler.fit_transform on the ENTIRE dataset before splitting. What went wrong?

   ANothing — scaling is deterministicBThe scaler's mean and std now include test rows, leaking test information into trainingCfit_transform only works on training data and will raise an errorDThe data will no longer be standardized

3. Q3.Why is cross_val_score(pipeline, ...) safer than scaling manually and then cross-validating the bare model?

   AIt's fasterBThe pipeline refits the scaler within each fold, so validation folds never influence preprocessingCIt uses more folds by defaultDIt automatically tunes hyperparameters

4. Q4.In a pipeline with a step named "knn", how do you refer to its n_neighbors in a GridSearchCV param grid?

   Aknn.n_neighborsBn_neighborsCknn__n_neighborsDknn->n_neighbors

5. Q5.What does ColumnTransformer add on top of Pipeline?

   AIt trains multiple models in parallelBIt applies different preprocessing to different subsets of columnsCIt removes columns with missing valuesDIt converts DataFrames to NumPy arrays

Exercise: Build a leak-proof pipeline for messy data

Create a synthetic 200-row DataFrame with two numeric columns (income, age — make the label depend on income), one categorical column (city), and about 15% missing values in income. Build a full pipeline — ColumnTransformer preprocessing plus a KNeighborsClassifier — and tune n_neighbors and weights with a small GridSearchCV. Report the best CV score and the final test accuracy.

Next up: before any pipeline runs, you have to understand your data — EDA and feature engineering, where most real-world accuracy is won.