{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11"
  },
  "colab": {
   "provenance": []
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cell-0000",
   "metadata": {},
   "source": [
    "# scikit-learn Pipelines & Workflow\n",
    "\n",
    "The estimator API, data leakage, and how Pipeline, ColumnTransformer, and GridSearchCV turn your workflow into one tunable, leak-proof object.\n",
    "\n",
    "*Part of the free [Machine Learning](https://ramadnsyh.dev/courses/machine-learning) course by [Muhammad Ramadiansyah](https://ramadnsyh.dev). This notebook is generated from the interactive lesson — [read it online](https://ramadnsyh.dev/courses/machine-learning/sklearn-pipelines).*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0001",
   "metadata": {},
   "source": [
    "You now know the workflow (split → fit → predict → evaluate) and you've seen\n",
    "that preprocessing like scaling must be fit on training data only. Doing that\n",
    "by hand gets error-prone fast — real datasets need imputation *and* scaling\n",
    "*and* encoding, each with its own fit-on-train rule. scikit-learn's answer is\n",
    "the **Pipeline**: bundle every step into a single object that behaves like\n",
    "one model. This lesson is the glue that holds the rest of the course\n",
    "together."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0002",
   "metadata": {},
   "source": [
    "## One interface: fit, predict, transform\n",
    "\n",
    "Everything in scikit-learn is an **estimator** sharing a tiny, consistent\n",
    "API:\n",
    "\n",
    "- `est.fit(X_train, y_train)` — learn from data. Models learn parameters;\n",
    "  preprocessors learn statistics (a scaler learns means, an encoder learns\n",
    "  categories).\n",
    "- `est.predict(X)` — for **models** (predictors): output labels or numbers.\n",
    "- `est.transform(X)` — for **transformers** (preprocessors): output a\n",
    "  modified copy of the data.\n",
    "- `est.fit_transform(X)` — fit then transform in one call (train data only!).\n",
    "\n",
    "Because every scaler, encoder, imputer, and model speaks this same language,\n",
    "you can swap a KNN for a random forest — or a MinMax scaler for a standard\n",
    "one — without changing the surrounding code. That uniformity is what makes\n",
    "pipelines possible."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0003",
   "metadata": {},
   "source": [
    "## Leakage: the silent score inflator\n",
    "\n",
    "**Data leakage** is when information from the test set sneaks into training.\n",
    "The model's scores look great, then reality disappoints. The most common\n",
    "leaks are mundane:\n",
    "\n",
    "- **Scaling with statistics from all rows** — the test set's mean has leaked\n",
    "  into training.\n",
    "- **Imputing missing values using the full dataset** — same problem.\n",
    "- **Tuning hyperparameters against the test set** — after enough peeks, the\n",
    "  test set is effectively memorized."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0004",
   "metadata": {},
   "source": [
    "> **The rule that prevents leakage**\n",
    "> \n",
    "> Fit anything that learns from data — scalers, imputers, encoders, models —\n",
    "> on the training split only. Then apply (transform) it, unchanged, to the\n",
    "> test split. Never call fit, or fit_transform, on test data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0005",
   "metadata": {},
   "source": [
    "## Pipeline: preprocessing and model as one estimator\n",
    "\n",
    "A `Pipeline` chains transformers and ends with a model. When you call\n",
    "`fit`, it fit-transforms each step on training data in sequence; when you\n",
    "call `predict` or `score`, it only *transforms* — the fit-on-train rule is\n",
    "enforced automatically:"
   ]
  },
  {
   "cell_type": "code",
   "id": "cell-0006",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "from sklearn.model_selection import train_test_split, cross_val_score\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "X, y = load_iris(return_X_y=True)\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.3, stratify=y, random_state=42\n",
    ")\n",
    "\n",
    "pipe = Pipeline([\n",
    "    (\"scaler\", StandardScaler()),\n",
    "    (\"knn\", KNeighborsClassifier(n_neighbors=5)),\n",
    "])\n",
    "\n",
    "pipe.fit(X_train, y_train)             # scaler fit on train, then knn fit\n",
    "print(f\"test accuracy: {pipe.score(X_test, y_test):.3f}\")\n",
    "\n",
    "# Cross-validation: 5 different train/validation splits, scaler refit each time\n",
    "scores = cross_val_score(pipe, X_train, y_train, cv=5)\n",
    "print(f\"5-fold CV    : {scores.round(3)}\")\n",
    "print(f\"mean ± std   : {scores.mean():.3f} ± {scores.std():.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0007",
   "metadata": {},
   "source": [
    "The `cross_val_score` call is doing something subtle and important. A single\n",
    "train/test split can be lucky or unlucky; **k-fold cross-validation** splits\n",
    "the training data into k parts, trains on k−1 and validates on the remaining\n",
    "one, rotating k times — like judging a student on both a midterm and a final\n",
    "instead of one exam. The mean of the k scores is a far more robust estimate.\n",
    "And because we passed the *pipeline*, the scaler is refit inside every fold —\n",
    "zero leakage, zero manual bookkeeping."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0008",
   "metadata": {},
   "source": [
    "## Mixed columns: ColumnTransformer\n",
    "\n",
    "Real tables mix numeric columns (scale them, impute with the median) and\n",
    "categorical columns (impute with the most frequent value, one-hot encode\n",
    "them). `ColumnTransformer` routes each group of columns through its own\n",
    "mini-pipeline:"
   ]
  },
  {
   "cell_type": "code",
   "id": "cell-0009",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder, StandardScaler\n",
    "\n",
    "rng = np.random.default_rng(42)\n",
    "n = 8\n",
    "df = pd.DataFrame({\n",
    "    \"age\":  [22, 35, np.nan, 58, 41, np.nan, 29, 63],\n",
    "    \"fare\": [7.9, 71.3, 8.1, 26.6, np.nan, 13.0, 30.1, 77.9],\n",
    "    \"sex\":  [\"male\", \"female\", \"female\", \"male\", \"male\", None, \"female\", \"male\"],\n",
    "    \"port\": [\"S\", \"C\", \"S\", \"S\", \"Q\", \"S\", \"C\", \"S\"],\n",
    "})\n",
    "print(df)\n",
    "\n",
    "numeric_pipe = Pipeline([\n",
    "    (\"imputer\", SimpleImputer(strategy=\"median\")),\n",
    "    (\"scaler\", StandardScaler()),\n",
    "])\n",
    "categorical_pipe = Pipeline([\n",
    "    (\"imputer\", SimpleImputer(strategy=\"most_frequent\")),\n",
    "    (\"onehot\", OneHotEncoder(handle_unknown=\"ignore\")),\n",
    "])\n",
    "\n",
    "preprocessor = ColumnTransformer([\n",
    "    (\"num\", numeric_pipe, [\"age\", \"fare\"]),\n",
    "    (\"cat\", categorical_pipe, [\"sex\", \"port\"]),\n",
    "])\n",
    "\n",
    "X_ready = preprocessor.fit_transform(df)\n",
    "print(\"\\\\nshape after preprocessing:\", X_ready.shape)\n",
    "print(preprocessor.get_feature_names_out())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0010",
   "metadata": {},
   "source": [
    "Four messy columns in; seven clean numeric columns out — missing values\n",
    "filled, numerics standardized, categories expanded into one-hot indicator\n",
    "columns. `handle_unknown=\"ignore\"` keeps predictions from crashing if a\n",
    "category appears at test time that training never saw."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0011",
   "metadata": {},
   "source": [
    "## Tuning the whole pipeline: GridSearchCV\n",
    "\n",
    "Hyperparameters (like KNN's `n_neighbors`) shouldn't be tuned against the\n",
    "test set. `GridSearchCV` tries every parameter combination using\n",
    "cross-validation *inside the training data*, then refits the best one. Name\n",
    "pipeline steps and address their parameters with a double underscore, as in\n",
    "`knn__n_neighbors`:"
   ]
  },
  {
   "cell_type": "code",
   "id": "cell-0012",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "X, y = load_iris(return_X_y=True)\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.3, stratify=y, random_state=42\n",
    ")\n",
    "\n",
    "pipe = Pipeline([\n",
    "    (\"scaler\", StandardScaler()),\n",
    "    (\"knn\", KNeighborsClassifier()),\n",
    "])\n",
    "\n",
    "param_grid = {\n",
    "    \"knn__n_neighbors\": [3, 5, 9, 15],\n",
    "    \"knn__weights\": [\"uniform\", \"distance\"],\n",
    "}\n",
    "\n",
    "search = GridSearchCV(pipe, param_grid, cv=3)\n",
    "search.fit(X_train, y_train)\n",
    "\n",
    "print(\"best params  :\", search.best_params_)\n",
    "print(f\"best CV score: {search.best_score_:.3f}\")\n",
    "print(f\"test accuracy: {search.score(X_test, y_test):.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0013",
   "metadata": {},
   "source": [
    "That's 4 × 2 = 8 combinations, each cross-validated 3 times — 24 fits, all\n",
    "leak-free because scaling happens inside each fold. On real projects the grid\n",
    "is bigger and you'd add `n_jobs=-1` to parallelize across CPU cores (and\n",
    "consider `RandomizedSearchCV` when the grid explodes).\n",
    "\n",
    "The complete modern workflow, then, in one breath: **split → build\n",
    "preprocessor + model into a pipeline → grid-search with CV on the training\n",
    "set → evaluate once on the test set.** This skeleton carries you through\n",
    "nearly every tabular ML problem."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0014",
   "metadata": {},
   "source": [
    "### 🏋️ Exercise — Build a leak-proof pipeline for messy data\n",
    "\n",
    "50).astype(int)\n",
    "\n",
    "# Punch some holes in the data\n",
    "income[rng.random(n) < 0.15] = np.nan\n",
    "df = pd.DataFrame({\"income\": income, \"age\": age, \"city\": city})\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    df, y, test_size=0.25, stratify=y, random_state=42\n",
    ")\n",
    "\n",
    "preprocessor = ColumnTransformer([\n",
    "    (\"num\", Pipeline([(\"imp\", SimpleImputer(strategy=\"median\")),\n",
    "                      (\"sc\", StandardScaler())]), [\"income\", \"age\"]),\n",
    "    (\"cat\", Pipeline([(\"imp\", SimpleImputer(strategy=\"most_frequent\")),\n",
    "                      (\"oh\", OneHotEncoder(handle_unknown=\"ignore\"))]), [\"city\"]),\n",
    "])\n",
    "\n",
    "pipe = Pipeline([(\"prep\", preprocessor), (\"algo\", KNeighborsClassifier())])\n",
    "\n",
    "grid = {\"algo__n_neighbors\": [5, 9, 15], \"algo__weights\": [\"uniform\", \"distance\"]}\n",
    "search = GridSearchCV(pipe, grid, cv=3).fit(X_train, y_train)\n",
    "\n",
    "print(\"best params  :\", search.best_params_)\n",
    "print(f\"best CV score: {search.best_score_:.3f}\")\n",
    "print(f\"test accuracy: {search.score(X_test, y_test):.3f}\")\n",
    "`}\n",
    ">\n",
    "Create a synthetic 200-row DataFrame with two numeric columns (`income`,\n",
    "`age` — make the label depend on income), one categorical column (`city`),\n",
    "and about 15% missing values in `income`. Build a full pipeline —\n",
    "ColumnTransformer preprocessing plus a `KNeighborsClassifier` — and tune\n",
    "`n_neighbors` and `weights` with a small `GridSearchCV`. Report the best CV\n",
    "score and the final test accuracy."
   ]
  },
  {
   "cell_type": "code",
   "id": "cell-0015",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Your solution here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0016",
   "metadata": {},
   "source": [
    "<details><summary>✅ Show solution</summary>\n",
    "\n",
    "```python\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder, StandardScaler\n",
    "\n",
    "rng = np.random.default_rng(7)\n",
    "n = 200\n",
    "income = rng.normal(50, 15, n)\n",
    "age = rng.normal(40, 12, n)\n",
    "city = rng.choice([\"north\", \"south\", \"east\"], n)\n",
    "y = (income + rng.normal(0, 8, n) > 50).astype(int)\n",
    "\n",
    "# Punch some holes in the data\n",
    "income[rng.random(n) < 0.15] = np.nan\n",
    "df = pd.DataFrame({\"income\": income, \"age\": age, \"city\": city})\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    df, y, test_size=0.25, stratify=y, random_state=42\n",
    ")\n",
    "\n",
    "preprocessor = ColumnTransformer([\n",
    "    (\"num\", Pipeline([(\"imp\", SimpleImputer(strategy=\"median\")),\n",
    "                      (\"sc\", StandardScaler())]), [\"income\", \"age\"]),\n",
    "    (\"cat\", Pipeline([(\"imp\", SimpleImputer(strategy=\"most_frequent\")),\n",
    "                      (\"oh\", OneHotEncoder(handle_unknown=\"ignore\"))]), [\"city\"]),\n",
    "])\n",
    "\n",
    "pipe = Pipeline([(\"prep\", preprocessor), (\"algo\", KNeighborsClassifier())])\n",
    "\n",
    "grid = {\"algo__n_neighbors\": [5, 9, 15], \"algo__weights\": [\"uniform\", \"distance\"]}\n",
    "search = GridSearchCV(pipe, grid, cv=3).fit(X_train, y_train)\n",
    "\n",
    "print(\"best params  :\", search.best_params_)\n",
    "print(f\"best CV score: {search.best_score_:.3f}\")\n",
    "print(f\"test accuracy: {search.score(X_test, y_test):.3f}\")\n",
    "```\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0017",
   "metadata": {},
   "source": [
    "Next up: before any pipeline runs, you have to understand your data — EDA\n",
    "and feature engineering, where most real-world accuracy is won."
   ]
  }
 ]
}