{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11"
  },
  "colab": {
   "provenance": []
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cell-0000",
   "metadata": {},
   "source": [
    "# What Is Machine Learning?\n",
    "\n",
    "Rules vs learning, the vocabulary of ML, and the golden rule — never evaluate a model on the data it trained on.\n",
    "\n",
    "*Part of the free [Machine Learning](https://ramadnsyh.dev/courses/machine-learning) course by [Muhammad Ramadiansyah](https://ramadnsyh.dev). This notebook is generated from the interactive lesson — [read it online](https://ramadnsyh.dev/courses/machine-learning/what-is-machine-learning).*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0001",
   "metadata": {},
   "source": [
    "Before touching any algorithm, you need three things: a clear picture of what\n",
    "\"learning from data\" actually means, the vocabulary to talk about it, and one\n",
    "non-negotiable habit — always keeping some data hidden from your model so you\n",
    "can measure how well it *really* performs. This lesson builds all three."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0002",
   "metadata": {},
   "source": [
    "## From writing rules to learning them\n",
    "\n",
    "Suppose you're asked to build a spam filter. The classical approach is to\n",
    "write rules by hand: *if the subject contains \"FREE!!!\", flag it; if the\n",
    "sender is unknown and there are more than three links, flag it…* This works\n",
    "until spammers change tactics, and every fix adds another brittle rule.\n",
    "\n",
    "Machine learning flips the recipe. Instead of writing the rules, you collect\n",
    "**examples** — thousands of emails already labeled *spam* or *not spam* — and\n",
    "let an algorithm find the patterns itself. The program's behavior is\n",
    "**learned from data** rather than hard-coded.\n",
    "\n",
    "That's the relationship between the two famous buzzwords: **artificial\n",
    "intelligence** is the broad goal of making machines behave intelligently\n",
    "(rule-based expert systems count too), while **machine learning** is the\n",
    "subfield where that behavior is learned from examples. Almost everything\n",
    "called \"AI\" today is machine learning underneath."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0003",
   "metadata": {},
   "source": [
    "## Three flavors of learning\n",
    "\n",
    "- **Supervised learning** — every example comes with the correct answer (a\n",
    "  *label*). The model learns to map inputs to answers: spam detection, house\n",
    "  price prediction, medical diagnosis. This is most of the course.\n",
    "- **Unsupervised learning** — no labels at all. The model looks for structure\n",
    "  on its own: grouping similar customers, compressing features. We'll get\n",
    "  there in the clustering and PCA module.\n",
    "- **Reinforcement learning** — an agent learns by acting and receiving\n",
    "  rewards, like a game-playing bot. Fascinating, but outside our scope here.\n",
    "\n",
    "Within supervised learning there are two main task types: **classification**\n",
    "(predict a category: spam / not spam) and **regression** (predict a number:\n",
    "tomorrow's temperature)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0004",
   "metadata": {},
   "source": [
    "## Samples, features, labels\n",
    "\n",
    "Data almost always arrives as a table, and ML has names for its parts:\n",
    "\n",
    "| Term | In the table | Convention |\n",
    "|---|---|---|\n",
    "| **Sample** (observation, instance) | one row | `n_samples` of them |\n",
    "| **Feature** (attribute, predictor) | one input column | matrix `X`, shape `(n_samples, n_features)` |\n",
    "| **Label** (target) | the column to predict | vector `y` |\n",
    "\n",
    "So \"train a model\" means: given `X` and `y`, find a function that maps a new\n",
    "row of features to a good prediction of its label."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0005",
   "metadata": {},
   "source": [
    "## The golden rule: never grade a model on its homework\n",
    "\n",
    "Here's a trap every beginner falls into. You fit a model, evaluate it *on the\n",
    "same data it trained on*, see 100% accuracy, and celebrate. But a model that\n",
    "memorizes its training data perfectly can still be useless on new data — and\n",
    "new data is the only thing we care about.\n",
    "\n",
    "Think of it like studying for an exam: if the exam questions are the exact\n",
    "homework problems you practiced, a perfect score proves you memorized the\n",
    "homework, not that you understand the subject. To measure understanding, the\n",
    "exam must contain **questions you've never seen**.\n",
    "\n",
    "In ML the fix is the **train/test split**: set aside a portion of the data\n",
    "(typically 20–30%), train only on the rest, and evaluate on the held-out\n",
    "part. Watch how dramatic the difference can be — this decision tree is\n",
    "allowed to grow as deep as it likes on noisy data:"
   ]
  },
  {
   "cell_type": "code",
   "id": "cell-0006",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_classification\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "# Noisy data: only 3 of 10 features matter, and 25% of labels are flipped\n",
    "X, y = make_classification(n_samples=400, n_features=10, n_informative=3,\n",
    "                           flip_y=0.25, random_state=42)\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.25, stratify=y, random_state=42\n",
    ")\n",
    "\n",
    "tree = DecisionTreeClassifier(random_state=42).fit(X_train, y_train)\n",
    "\n",
    "print(f\"train accuracy: {tree.score(X_train, y_train):.3f}\")\n",
    "print(f\"test accuracy : {tree.score(X_test, y_test):.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0007",
   "metadata": {},
   "source": [
    "Perfect on training data, mediocre on test data. If we had evaluated on the\n",
    "training set, we would have believed we had a flawless model.\n",
    "\n",
    "A few details in that split call matter:\n",
    "\n",
    "- `test_size=0.25` — hold out 25% of the rows for testing.\n",
    "- `stratify=y` — keep the class proportions the same in both splits, so\n",
    "  neither set is accidentally easier.\n",
    "- `random_state=42` — make the shuffle reproducible."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0008",
   "metadata": {},
   "source": [
    "## Overfitting vs underfitting\n",
    "\n",
    "The gap you just saw has a name. A model **overfits** when it learns the\n",
    "training data too well — noise, quirks, and all — so its training score is\n",
    "high but its test score is much lower. It memorized the homework.\n",
    "\n",
    "The opposite failure is **underfitting**: the model is too simple to capture\n",
    "the real pattern, so *both* scores are low. Between the two lies the sweet\n",
    "spot, and controlling model complexity is how you find it:"
   ]
  },
  {
   "cell_type": "code",
   "id": "cell-0009",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_classification\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "X, y = make_classification(n_samples=400, n_features=10, n_informative=3,\n",
    "                           flip_y=0.25, random_state=42)\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.25, stratify=y, random_state=42\n",
    ")\n",
    "\n",
    "for depth in [1, 3, 5, None]:\n",
    "    tree = DecisionTreeClassifier(max_depth=depth, random_state=42)\n",
    "    tree.fit(X_train, y_train)\n",
    "    name = f\"depth={depth}\"\n",
    "    print(f\"{name:12s} train={tree.score(X_train, y_train):.3f}  \"\n",
    "          f\"test={tree.score(X_test, y_test):.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0010",
   "metadata": {},
   "source": [
    "Depth 1 underfits (both scores low), unlimited depth overfits (huge gap), and\n",
    "a moderate depth does best on the test set. You'll see this pattern in every\n",
    "model family for the rest of the course."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0011",
   "metadata": {},
   "source": [
    "> **The test set is sacred**\n",
    "> \n",
    "> Use the test set once, at the end, to estimate real-world performance. If you\n",
    "> peek at it repeatedly while tweaking your model, it silently becomes part of\n",
    "> training and its score stops being trustworthy. Later lessons introduce\n",
    "> cross-validation for safe, repeated evaluation during development."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0012",
   "metadata": {},
   "source": [
    "## The basic workflow\n",
    "\n",
    "Every supervised project in this course follows the same four beats:\n",
    "\n",
    "1. **Split** — `train_test_split` before anything else touches the data.\n",
    "2. **Fit** — `model.fit(X_train, y_train)` learns patterns from training data\n",
    "   only.\n",
    "3. **Predict** — `model.predict(X_test)` produces answers for unseen rows.\n",
    "4. **Evaluate** — compare predictions against `y_test` with a metric such as\n",
    "   accuracy.\n",
    "\n",
    "Everything else — preprocessing, feature engineering, hyperparameter tuning —\n",
    "is elaboration on this skeleton. Memorize the beats; the next lessons add the\n",
    "instruments."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0013",
   "metadata": {},
   "source": [
    "### 🏋️ Exercise — Watch overfitting grow with noise\n",
    "\n",
    "Using the unlimited-depth decision tree from this lesson, generate three\n",
    "datasets with `make_classification` at noise levels `flip_y = 0.0`, `0.1`,\n",
    "and `0.3` (keep everything else the same). For each, print the train\n",
    "accuracy, test accuracy, and the gap between them. How does label noise\n",
    "affect how badly the tree overfits?"
   ]
  },
  {
   "cell_type": "code",
   "id": "cell-0014",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Your solution here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0015",
   "metadata": {},
   "source": [
    "<details><summary>✅ Show solution</summary>\n",
    "\n",
    "```python\n",
    "from sklearn.datasets import make_classification\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "for noise in [0.0, 0.1, 0.3]:\n",
    "    X, y = make_classification(n_samples=400, n_features=10, n_informative=3,\n",
    "                               flip_y=noise, random_state=42)\n",
    "    X_train, X_test, y_train, y_test = train_test_split(\n",
    "        X, y, test_size=0.25, stratify=y, random_state=42\n",
    "    )\n",
    "    tree = DecisionTreeClassifier(random_state=42).fit(X_train, y_train)\n",
    "    gap = tree.score(X_train, y_train) - tree.score(X_test, y_test)\n",
    "    print(f\"flip_y={noise:.1f}  train={tree.score(X_train, y_train):.3f}  \"\n",
    "          f\"test={tree.score(X_test, y_test):.3f}  gap={gap:.3f}\")\n",
    "```\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-0016",
   "metadata": {},
   "source": [
    "Next up: your first real algorithm — K-Nearest Neighbors, a classifier so\n",
    "intuitive you already use it in daily life."
   ]
  }
 ]
}