{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.11" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "id": "cell-0000", "metadata": {}, "source": [ "# Collaborative Filtering\n", "\n", "Learn taste from behavior alone — build the user-item ratings matrix, predict missing ratings with item-item similarity, and uncover latent factors with SVD.\n", "\n", "*Part of the free [Machine Learning](https://ramadnsyh.dev/courses/machine-learning) course by [Muhammad Ramadiansyah](https://ramadnsyh.dev). This notebook is generated from the interactive lesson — [read it online](https://ramadnsyh.dev/courses/machine-learning/collaborative-filtering).*" ] }, { "cell_type": "markdown", "id": "cell-0001", "metadata": {}, "source": [ "Content-based filtering needs good item descriptions. Collaborative filtering\n", "needs none — it learns purely from *behavior*: who rated what, and how. The\n", "core bet is that if you and I rated ten movies the same way, my opinion of an\n", "eleventh movie is useful evidence about yours. In this lesson you'll build the\n", "ratings matrix, predict missing entries with neighborhood methods, and then\n", "compress the whole thing into latent \"taste\" factors with SVD." ] }, { "cell_type": "markdown", "id": "cell-0002", "metadata": {}, "source": [ "## The user-item matrix — mostly holes\n", "\n", "Everything in collaborative filtering starts from one object: a matrix with\n", "one row per user, one column per item, and ratings in the cells. Its defining\n", "property is **sparsity** — almost every cell is empty, because no one rates\n", "more than a sliver of the catalog. (Netflix-scale matrices are over 99%\n", "empty; recommending *is* filling in the blanks.)" ] }, { "cell_type": "code", "id": "cell-0003", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import pandas as pd\n", "\n", "ratings = pd.DataFrame({\n", " \"user\": [\"Andi\",\"Andi\",\"Andi\",\"Andi\",\n", " \"Budi\",\"Budi\",\"Budi\",\"Budi\",\n", " \"Citra\",\"Citra\",\"Citra\",\"Citra\",\n", " \"Dewi\",\"Dewi\",\"Dewi\",\"Dewi\",\n", " \"Eko\",\"Eko\",\"Eko\",\n", " \"Fira\",\"Fira\",\"Fira\",\"Fira\",\"Fira\"],\n", " \"movie\": [\"The Matrix\",\"John Wick\",\"Mad Max\",\"Titanic\",\n", " \"The Matrix\",\"John Wick\",\"Titanic\",\"The Notebook\",\n", " \"Titanic\",\"The Notebook\",\"La La Land\",\"The Matrix\",\n", " \"The Notebook\",\"La La Land\",\"Titanic\",\"John Wick\",\n", " \"The Matrix\",\"Mad Max\",\"La La Land\",\n", " \"John Wick\",\"Titanic\",\"La La Land\",\"The Notebook\",\"Mad Max\"],\n", " \"rating\": [5,4,5,1, 4,5,2,1, 5,4,5,2, 5,4,4,1, 5,4,2, 2,4,5,4,1],\n", "})\n", "\n", "R = ratings.pivot_table(index=\"user\", columns=\"movie\", values=\"rating\")\n", "print(R.to_string(), \"\\\\n\")\n", "\n", "sparsity = R.isna().sum().sum() / R.size\n", "print(f\"Cells: {R.size}, empty: {R.isna().sum().sum()} -> sparsity = {sparsity:.0%}\")" ] }, { "cell_type": "markdown", "id": "cell-0004", "metadata": {}, "source": [ "Squint at the matrix and two taste groups jump out: Andi, Budi, and Eko love\n", "the action films; Citra, Dewi, and Fira love the romances. The `NaN`s are\n", "exactly the questions a recommender must answer — should Eko watch\n", "*John Wick*?" ] }, { "cell_type": "markdown", "id": "cell-0005", "metadata": {}, "source": [ "## Neighborhood methods: user-based vs item-based\n", "\n", "The classic (\"memory-based\") approach predicts a missing rating from similar\n", "rows or similar columns:\n", "\n", "- **User-based CF** — find users whose rating vectors resemble yours, and\n", " average *their* ratings of the target item. \"People like you loved this.\"\n", "- **Item-based CF** — find items whose rating *columns* resemble the target\n", " item's, and average *your* ratings of those. \"You loved similar movies.\"\n", "\n", "Item-based usually wins in production, for a practical reason: **item\n", "similarities are stable**. A catalog has fewer items than users, items\n", "accumulate many ratings each, and their similarity profile barely changes\n", "day-to-day — so the item-item matrix can be precomputed offline. User tastes\n", "shift constantly and there are millions of users, making user-user similarity\n", "expensive and stale. (Amazon's original \"customers who bought X also bought Y\"\n", "was exactly precomputed item-based CF.)\n", "\n", "Let's do item-based on our small matrix: compute cosine similarity between\n", "item columns, then predict Eko's missing ratings as similarity-weighted\n", "averages of the ratings he *did* give." ] }, { "cell_type": "code", "id": "cell-0006", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import pandas as pd\n", "from sklearn.metrics.pairwise import cosine_similarity\n", "\n", "ratings = pd.DataFrame({\n", " \"user\": [\"Andi\",\"Andi\",\"Andi\",\"Andi\",\"Budi\",\"Budi\",\"Budi\",\"Budi\",\n", " \"Citra\",\"Citra\",\"Citra\",\"Citra\",\"Dewi\",\"Dewi\",\"Dewi\",\"Dewi\",\n", " \"Eko\",\"Eko\",\"Eko\",\"Fira\",\"Fira\",\"Fira\",\"Fira\",\"Fira\"],\n", " \"movie\": [\"The Matrix\",\"John Wick\",\"Mad Max\",\"Titanic\",\n", " \"The Matrix\",\"John Wick\",\"Titanic\",\"The Notebook\",\n", " \"Titanic\",\"The Notebook\",\"La La Land\",\"The Matrix\",\n", " \"The Notebook\",\"La La Land\",\"Titanic\",\"John Wick\",\n", " \"The Matrix\",\"Mad Max\",\"La La Land\",\n", " \"John Wick\",\"Titanic\",\"La La Land\",\"The Notebook\",\"Mad Max\"],\n", " \"rating\": [5,4,5,1, 4,5,2,1, 5,4,5,2, 5,4,4,1, 5,4,2, 2,4,5,4,1],\n", "})\n", "R = ratings.pivot_table(index=\"user\", columns=\"movie\", values=\"rating\")\n", "\n", "# Item-item cosine similarity (missing = 0 for the similarity step)\n", "sim = pd.DataFrame(cosine_similarity(R.fillna(0).T),\n", " index=R.columns, columns=R.columns)\n", "print(\"Most similar to John Wick:\")\n", "print(sim[\"John Wick\"].drop(\"John Wick\").sort_values(ascending=False).round(2).to_string(), \"\\\\n\")\n", "\n", "def predict(user, movie):\n", " rated = R.loc[user].dropna() # movies this user rated\n", " w = sim.loc[movie, rated.index] # similarity to each\n", " return (w * rated).sum() / w.sum() # weighted average\n", "\n", "def recommend(user, k=3):\n", " unseen = R.columns[R.loc[user].isna()]\n", " preds = pd.Series({m: predict(user, m) for m in unseen})\n", " return preds.sort_values(ascending=False).head(k)\n", "\n", "print(\"Predicted ratings for Eko's unseen movies:\")\n", "print(recommend(\"Eko\").round(2).to_string())" ] }, { "cell_type": "markdown", "id": "cell-0007", "metadata": {}, "source": [ "*John Wick*'s nearest neighbors are the other action films — learned from\n", "ratings alone, with zero genre metadata. And the prediction for Eko says\n", "exactly what intuition does: recommend *John Wick* (he loved the similar\n", "action films), not *The Notebook*." ] }, { "cell_type": "markdown", "id": "cell-0008", "metadata": {}, "source": [ "## Matrix factorization: latent taste factors\n", "\n", "Neighborhood methods compare raw rows and columns. **Matrix factorization**\n", "goes deeper: assume each user and each item can be described by a handful of\n", "hidden numbers — *latent factors* — such that a rating is roughly the dot\n", "product of the user's factor vector and the item's. With movies, factors often\n", "end up meaning things like \"action vs romance\" or \"mainstream vs arthouse\",\n", "even though nobody labeled them. Approximating our 6×6 matrix with just 2\n", "factors:" ] }, { "cell_type": "code", "id": "cell-0009", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import pandas as pd\n", "from sklearn.decomposition import TruncatedSVD\n", "\n", "ratings = pd.DataFrame({\n", " \"user\": [\"Andi\",\"Andi\",\"Andi\",\"Andi\",\"Budi\",\"Budi\",\"Budi\",\"Budi\",\n", " \"Citra\",\"Citra\",\"Citra\",\"Citra\",\"Dewi\",\"Dewi\",\"Dewi\",\"Dewi\",\n", " \"Eko\",\"Eko\",\"Eko\",\"Fira\",\"Fira\",\"Fira\",\"Fira\",\"Fira\"],\n", " \"movie\": [\"The Matrix\",\"John Wick\",\"Mad Max\",\"Titanic\",\n", " \"The Matrix\",\"John Wick\",\"Titanic\",\"The Notebook\",\n", " \"Titanic\",\"The Notebook\",\"La La Land\",\"The Matrix\",\n", " \"The Notebook\",\"La La Land\",\"Titanic\",\"John Wick\",\n", " \"The Matrix\",\"Mad Max\",\"La La Land\",\n", " \"John Wick\",\"Titanic\",\"La La Land\",\"The Notebook\",\"Mad Max\"],\n", " \"rating\": [5,4,5,1, 4,5,2,1, 5,4,5,2, 5,4,4,1, 5,4,2, 2,4,5,4,1],\n", "})\n", "R = ratings.pivot_table(index=\"user\", columns=\"movie\", values=\"rating\")\n", "\n", "svd = TruncatedSVD(n_components=2, random_state=42)\n", "user_factors = svd.fit_transform(R.fillna(0)) # 6 users x 2 factors\n", "item_factors = svd.components_ # 2 factors x 6 movies\n", "\n", "print(\"Item factors (movies as 2 hidden numbers):\")\n", "print(pd.DataFrame(item_factors, index=[\"factor 1\", \"factor 2\"],\n", " columns=R.columns).round(2).to_string(), \"\\\\n\")\n", "\n", "recon = pd.DataFrame(user_factors @ item_factors,\n", " index=R.index, columns=R.columns)\n", "print(\"Reconstructed matrix (blanks now filled):\")\n", "print(recon.round(1).to_string(), \"\\\\n\")\n", "print(\"Eko + John Wick was missing; SVD estimates:\",\n", " round(recon.loc[\"Eko\", \"John Wick\"], 2))" ] }, { "cell_type": "markdown", "id": "cell-0010", "metadata": {}, "source": [ "Look at the item factor table: one factor loads on the action films, the other\n", "on the romances — SVD rediscovered genre from ratings alone. The\n", "reconstruction fills every blank with a consistent estimate, and it compresses\n", "36 cells into 24 numbers; on real data, 100 factors can summarize millions of\n", "users. (Production systems use factorization variants that fit *only the\n", "observed cells* rather than treating blanks as zeros, plus regularization —\n", "the idea is the same.)" ] }, { "cell_type": "markdown", "id": "cell-0011", "metadata": {}, "source": [ "> **Explicit vs implicit feedback**\n", "> \n", "> Star ratings are **explicit** feedback: rare, but unambiguous. Most real\n", "> signal is **implicit**: clicks, watch time, purchases, skips. Implicit data is\n", "> abundant but one-sided — a click means interest, but no click doesn't mean\n", "> dislike. Implicit-feedback models therefore predict *confidence-weighted\n", "> preference* rather than a rating, and they power most modern recommenders." ] }, { "cell_type": "markdown", "id": "cell-0012", "metadata": {}, "source": [ "## Cold starts, hybrids, and how to evaluate\n", "\n", "Collaborative filtering's Achilles heel is the **cold start**: a new user has\n", "an empty row (nothing to match on), and a new item has an empty column (nobody\n", "can \"collaborate\" it into recommendations — no matter how good it is). Note\n", "the symmetry with the previous lessons: popularity needs no user history, and\n", "content-based handles brand-new items. So production systems are **hybrids**\n", "that route between families:" ] }, { "cell_type": "markdown", "id": "cell-0013", "metadata": {}, "source": [ "```text\n", "recommend(user, movie):\n", " if not logged in or brand-new account:\n", " popularity / weighted-rating charts # lesson 1\n", " else:\n", " candidates = top-30 most content-similar # lesson 2\n", " drop what the user already watched\n", " score candidates with collaborative model # this lesson\n", " return the top 10\n", "```" ] }, { "cell_type": "markdown", "id": "cell-0014", "metadata": {}, "source": [ "How do you know any of it works? Offline, hold out ratings as a test set —\n", "hide 20% of each user's ratings, train on the rest, and check the held-out\n", "ones. Two styles of metric:\n", "\n", "- **Rating accuracy** — RMSE/MAE between predicted and true held-out ratings.\n", "- **Ranking quality** — **precision@k**: of the top-k items you recommended,\n", " what fraction did the user actually like (e.g. rated 4+)? Ranking metrics\n", " match the product better: users see a top-10 list, not your rating estimate.\n", "\n", "The final verdict, though, always comes from **online A/B tests** — does the\n", "new recommender actually increase watches, saves, or purchases?" ] }, { "cell_type": "markdown", "id": "cell-0015", "metadata": {}, "source": [ "## Scaling up: MovieLens in Colab\n", "\n", "Our 6×6 matrix fits on a slide; the classic benchmark is **MovieLens 100k**\n", "(100,000 ratings, 943 users, 1,682 movies). The same code scales straight up —\n", "run this in Colab:" ] }, { "cell_type": "code", "id": "cell-0016", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import pandas as pd\n", "from sklearn.decomposition import TruncatedSVD\n", "\n", "# Download MovieLens 100k\n", "!wget -q https://files.grouplens.org/datasets/movielens/ml-100k.zip\n", "!unzip -q -o ml-100k.zip\n", "\n", "ratings = pd.read_csv(\"ml-100k/u.data\", sep=\"\\t\",\n", " names=[\"user_id\", \"movie_id\", \"rating\", \"timestamp\"])\n", "movies = pd.read_csv(\"ml-100k/u.item\", sep=\"|\", encoding=\"latin-1\",\n", " usecols=[0, 1], names=[\"movie_id\", \"title\"])\n", "\n", "R = ratings.pivot_table(index=\"user_id\", columns=\"movie_id\", values=\"rating\")\n", "print(f\"Matrix: {R.shape[0]} users x {R.shape[1]} movies, \"\n", " f\"sparsity = {R.isna().sum().sum() / R.size:.1%}\")\n", "\n", "# 50 latent factors\n", "svd = TruncatedSVD(n_components=50, random_state=42)\n", "user_factors = svd.fit_transform(R.fillna(0))\n", "recon = pd.DataFrame(user_factors @ svd.components_,\n", " index=R.index, columns=R.columns)\n", "\n", "# Top-10 unseen movies for user 1\n", "user = 1\n", "unseen = R.columns[R.loc[user].isna()]\n", "top10 = recon.loc[user, unseen].sort_values(ascending=False).head(10)\n", "print(movies.set_index(\"movie_id\").loc[top10.index, \"title\"].to_string())" ] }, { "cell_type": "markdown", "id": "cell-0017", "metadata": {}, "source": [ "For a proper project, add the evaluation loop: split ratings into train/test\n", "per user, fit on train, and report precision@10 against the held-out likes." ] }, { "cell_type": "markdown", "id": "cell-0018", "metadata": {}, "source": [ "### 🏋️ Exercise — Recommend for the romance crowd\n", "\n", "Using the item-based `predict()` and `recommend()` functions from the lesson,\n", "generate recommendations for **Citra** and **Fira**. Are the predicted ratings\n", "for the action movies high or low — and why does that make sense given the\n", "item-item similarity table? Then check whether the SVD reconstruction agrees\n", "with the neighborhood method for the same missing cells." ] }, { "cell_type": "code", "id": "cell-0019", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "# Your solution here" ] }, { "cell_type": "markdown", "id": "cell-0020", "metadata": {}, "source": [ "
✅ Show solution\n", "\n", "```python\n", "# With the item-based predict/recommend functions from the lesson:\n", "print(\"Citra:\")\n", "print(recommend(\"Citra\").round(2).to_string())\n", "\n", "print(\"\\\\nFira:\")\n", "print(recommend(\"Fira\").round(2).to_string())\n", "\n", "print(\"\\\\nSpot-check single predictions:\")\n", "print(\"Citra + Mad Max :\", round(predict(\"Citra\", \"Mad Max\"), 2))\n", "print(\"Fira + The Matrix:\", round(predict(\"Fira\", \"The Matrix\"), 2))\n", "\n", "# Both users' predicted ratings for action titles come out low-to-middling,\n", "# dragged down because their high ratings sit on romance films that have low\n", "# similarity to the action columns. The recommender correctly declines to\n", "# push John Wick on Citra with a high score.\n", "```\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "cell-0021", "metadata": {}, "source": [ "That wraps the recommender-systems module — you now have all three classic\n", "families and the hybrid pattern that ties a real production system together." ] } ] }