vault

Building a Simple Recommender

21 min read

Building a Simple Recommender

A progressive, hands-on walkthrough that builds a recommender from the simplest possible starting point — a non-personalized popularity ranking — up through content-based filtering, item-item collaborative filtering, implicit-feedback matrix factorization, and modern feature-driven hybrid architectures. Each stage adds capability and resolves a specific failure of the previous stage. The walkthrough is intentionally pedagogical: it favours small amounts of code that you can paste into a notebook and inspect over production-grade systems with thousands of lines of infrastructure. The deeper lesson — and the single most-violated principle in recommender engineering — is that you should not skip stages. A popularity baseline is not a strawman; it is the floor every fancier model must beat. Many production teams discover, after building a 50-million-parameter deep model, that it loses to a tuned popularity ranker on their data. Working through the stages in order builds the diagnostic intuition needed to know when each step actually helps.

1. The Discipline of Working Up the Stack

Recommender engineering has a chronic temptation to start at the top of the algorithmic complexity stack — pick a deep neural architecture from a recent paper, train it, ship it. This is almost always wrong, for three reasons.

You don’t know what good looks like. Without a baseline, the only metric you can compute is “the new model’s offline NDCG.” That number is meaningless in isolation. Is 0.42 good? Is it bad? You cannot tell unless you have something to compare against. Stage 0 (popularity) gives you a number to beat. Stage 1 (content-based) gives you a stronger one. Each stage’s offline metric becomes the floor for the next.

You don’t know which improvement comes from which design choice. A deep model has many simultaneous design choices: architecture, regularization, learning rate, embedding dimension, negative sampling. If your initial deep model loses to popularity, you cannot tell which choice is responsible. Building up stage-by-stage isolates each design decision so you can attribute gains and losses.

You don’t know when to stop. Many production systems stop at stage 3 (matrix factorization with implicit feedback) and never need stage 4 or 5. Building stages 4 and 5 takes weeks of engineering effort plus ongoing operational cost. If stage 3 is good enough, the additional work is wasted. The decision tree at the end of this note (§9) helps with this.

This walkthrough assumes a dataset with at minimum a (user_id, item_id, rating) table for explicit-feedback systems or a (user_id, item_id, timestamp) table for implicit feedback. The code uses Python with scikit-learn, scipy, pandas, and a few specialized libraries (implicit, lightfm); it should run on any modern laptop for datasets up to about 10 million interactions.

2. Stage 0 — Popularity Baseline

The simplest recommender that works. No personalization. Same recommendations for everyone.

import pandas as pd
 
# ratings.csv has columns: user_id, item_id, rating, timestamp
ratings = pd.read_csv("ratings.csv")
 
# Compute mean rating and number of ratings per item.
stats = ratings.groupby("item_id").agg(
    mean_rating=("rating", "mean"),
    n_ratings=("rating", "count"),
)
 
# Bayesian shrinkage: pull items with few ratings toward the global mean.
# Without this, items with two glowing reviews can outrank items with thousands
# of moderate reviews — a noisy and bad-faith ranking.
C = stats["mean_rating"].mean()           # global mean rating
m = stats["n_ratings"].quantile(0.75)     # "minimum reviews" threshold
 
stats["score"] = (stats["n_ratings"] * stats["mean_rating"]
                  + m * C) / (stats["n_ratings"] + m)
 
top = stats.sort_values("score", ascending=False).head(10)

What this code does, step by step:

  • Reads the ratings into a pandas DataFrame.
  • Aggregates per item: the mean rating and the number of ratings. These are the two ingredients of a popularity ranking.
  • Computes C, the global mean rating across all items. This is the prior the shrinkage will pull toward.
  • Picks m, a “minimum interactions” threshold (here the 75th percentile of n_ratings). Items with at least m ratings will be dominated by their own mean; items with fewer ratings will be pulled toward C.
  • Computes the shrunk score score = (n · r + m · C) / (n + m), the Bayesian shrinkage formula used in the IMDB Top 250. With n = 0, the score is C; with n large, the score approaches r. The transition is smooth.
  • Sorts by score and returns the top 10.

The Bayesian shrinkage trick is non-obvious and important. Without it, the highest-scoring items are inevitably ones with very few ratings (the variance is high; some will get lucky), and the recommendation list is dominated by noise. With it, the ranking favours items with both a good rating and enough support to be confident in that rating.

What stage 0 solves: the recommender is up; the system has a usable ranking; new users get something reasonable.

What stage 0 does not solve: zero personalization (everyone sees the same list); no use of item content; no use of co-occurrence patterns. New items still have no scores until they accumulate ratings.

When to stop here: if your platform’s traffic is dominated by exploration (“what’s trending?”), if user interactions are too sparse to support personalization, or if a popularity ranking just plain works for the use case (some news platforms operate this way), stage 0 may be enough.

3. Stage 1 — Content-Based Filtering with TF-IDF

Add personalization based on item features. The recommender now looks at what each user has liked in the past and matches future recommendations to those features. The principal benefit over stage 0 is that recommendations differ per user; the secondary benefit is that new items are no longer invisible (because they are featurizable from day one).

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
 
items = pd.read_csv("items.csv")  # columns: item_id, title, description
 
# Build the TF-IDF matrix over item descriptions.
# stop_words removes "the", "a", etc. that carry no topical content.
# max_features caps the vocabulary at the 20k most-common terms.
# ngram_range=(1, 2) includes unigrams and bigrams.
tfidf = TfidfVectorizer(
    stop_words="english",
    max_features=20_000,
    ngram_range=(1, 2),
)
M = tfidf.fit_transform(items["description"].fillna(""))   # shape: n_items × n_terms
 
# Index lookup so we can map item_id to row index in M.
item_id_to_idx = {item_id: i for i, item_id in enumerate(items["item_id"])}
 
def recommend_for_user(user_id, top_n=10):
    # Get items this user has rated highly (>= 4 stars).
    liked = ratings[(ratings.user_id == user_id) & (ratings.rating >= 4)]["item_id"]
    if len(liked) == 0:
        return None  # cold user — fall back to stage 0 popularity
 
    # User profile = mean of liked items' TF-IDF vectors.
    liked_idx = [item_id_to_idx[iid] for iid in liked]
    user_profile = M[liked_idx].mean(axis=0)   # shape: 1 × n_terms
 
    # Score every item by cosine similarity to the user profile.
    scores = cosine_similarity(np.asarray(user_profile), M).flatten()
 
    # Mask out items the user has already liked, so we recommend genuinely new items.
    scores[liked_idx] = -np.inf
 
    # Top-N items by score.
    top_indices = scores.argsort()[::-1][:top_n]
    return items.iloc[top_indices]

What this code does:

  • Builds a TF-IDF vectorizer over item descriptions. TF-IDF (Term Frequency–Inverse Document Frequency) is defined precisely in Content-Based Filtering §3.1; in brief, it weights each term in each document inversely to how common the term is across the entire collection, so distinctive terms float to the top of each document’s vector.
  • For each user, computes a user profile vector as the mean of the TF-IDF vectors of items they liked. The user profile lives in the same TF-IDF space as the items.
  • Scores every item by cosine similarity to the user profile. Cosine measures the angle between vectors and ignores magnitude, which is what we want for sparse text vectors.
  • Masks already-liked items so the recommender does not just recommend things the user has already seen.

What stage 1 solves: personalization (different users get different lists); new-item cold start (a new item gets a TF-IDF vector from day one and can be scored immediately).

What stage 1 does not solve: affinities that are not visible in the features. If two items have no shared text but are loved by the same users, content-based filtering cannot connect them. This is the filter bubble of content-based methods — see Content-Based Filtering §7.

When to stop here: if your items have rich textual or feature descriptions, your scale is modest (under a million items), and user behaviour is well-explained by item features, stage 1 may be sufficient.

4. Stage 2 — Item-Item Collaborative Filtering

Switch to learning affinities from user co-occurrence patterns. The Amazon-2003 algorithm. The recommender now looks at “users who liked X also liked Y” relationships, capturing affinities that are invisible in features.

from scipy.sparse import csr_matrix
from sklearn.metrics.pairwise import cosine_similarity
 
# Build the sparse user-item interaction matrix.
# Treat ratings >= 4 as binary positive interactions (implicit framing).
positive = ratings[ratings.rating >= 4]
user_idx = pd.factorize(positive.user_id)[0]    # contiguous integer user ids
item_idx = pd.factorize(positive.item_id)[0]    # contiguous integer item ids
n_users = user_idx.max() + 1
n_items = item_idx.max() + 1
 
R = csr_matrix(
    ([1.0] * len(positive), (user_idx, item_idx)),
    shape=(n_users, n_items),
)
 
# Item-item cosine similarity, computed on the user vectors.
# Note: this is n_items × n_items; fine for ~10⁴ items, blows up at 10⁵ unless
# you switch to approximate methods.
item_sim = cosine_similarity(R.T, dense_output=False)
 
# Recommendation for user u: weighted sum over items they liked.
def recommend(u_idx, top_n=10):
    user_history = R[u_idx]                          # 1 × n_items, sparse
    scores = user_history @ item_sim                 # 1 × n_items, sparse
    scores = scores.toarray().flatten()
    # Mask seen items so the recommender doesn't return them.
    scores[user_history.toarray().flatten() > 0] = -np.inf
    return scores.argsort()[::-1][:top_n]

What this code does:

  • Constructs a sparse m × n binary matrix R where R[u, i] = 1 if user u rated item i at least 4 stars, else 0. The conversion uses csr_matrix (Compressed Sparse Row format) for efficient row access.
  • Computes the item-item cosine similarity matrix item_sim of shape n × n. The matrix entry item_sim[i, j] is the cosine similarity between the two columns of R corresponding to items i and j — high when the two items are liked by overlapping sets of users.
  • For a target user, computes the recommendation scores as the matrix product of the user’s history row with the item-similarity matrix. The result is a vector of scores, one per item, where each item’s score is the sum of similarities to items in the user’s history.

The matrix product user_history @ item_sim is the key computation. For each item j in the catalog, the score is Σ_i user_history[i] · item_sim[i, j] = Σ_{i ∈ liked} item_sim[i, j], which is exactly the prediction formula in Item-Item Collaborative Filtering §4 (without the normalization, which is unnecessary for ranking).

What stage 2 solves: taste-based affinities that content-based methods miss; explainable recommendations (“because you liked X, here is Y, which other people who liked X also liked”).

What stage 2 does not solve: sparsity (the cosine on user vectors with little overlap is noisy without shrinkage); cold items (a brand-new item has no co-purchase history); higher-order patterns.

When to stop here: if your user-item matrix has reasonable density (more than a few items per user on average), if explainability matters, and if the catalog is small enough (under a million items) that the n × n similarity matrix is tractable, stage 2 may be sufficient.

5. Stage 3 — Implicit-Feedback Matrix Factorization (iALS)

Compress the matrix into latent factors. This stage handles sparsity by forcing the model to explain observations with a low-dimensional latent space, and it scales much further than stage 2 because the factor matrices are O(m + n), not O(n²). This is the algorithm that won the Netflix Prize era and that powers most production recommenders that are not feature-driven deep models.

The cleanest implementation uses the implicit library by Ben Frederickson, which is a fast C implementation of the Hu, Koren & Volinsky 2008 iALS algorithm:

import implicit
from scipy.sparse import csr_matrix
 
# Reuse R from stage 2.
# Apply confidence weighting: c_ui = 1 + alpha · r_ui.
# For binary R, this is c_ui = 1 + alpha if interacted else 1.
alpha = 40.0
R_conf = R * alpha   # element-wise scaling
 
model = implicit.als.AlternatingLeastSquares(
    factors=64,             # latent dimension k
    regularization=0.01,    # L2 lambda
    iterations=15,          # ALS alternation count
    use_gpu=False,          # set True if CUDA is available
)
model.fit(R_conf)           # learns model.user_factors and model.item_factors
 
# Recommend for a user.
recs = model.recommend(
    userid=7,
    user_items=R[7],
    N=10,
    filter_already_liked_items=True,
)
# recs is an array of (item_idx, score) tuples.

What this code does:

  • Constructs the confidence matrix R_conf by element-wise scaling. Recall from Explicit vs Implicit Feedback §3 that the iALS confidence formula is c_{u,i} = 1 + α · r_{u,i}; the +1 is added implicitly by the library, so what we pass in is α · r_{u,i}.
  • Constructs an AlternatingLeastSquares model with 64 latent factors, L2 regularization 0.01, and 15 ALS iterations. These are reasonable defaults; tune empirically against held-out NDCG@10.
  • Calls model.fit(R_conf), which runs the ALS optimization and produces model.user_factors (shape n_users × 64) and model.item_factors (shape n_items × 64).
  • Calls model.recommend(...) to get the top 10 items for user 7. The library handles the masking of already-liked items via filter_already_liked_items=True.

The whole thing is roughly 15 lines and produces a production-quality recommender. The implicit library handles the algorithmic complexity (the Hu/Koren/Volinsky reformulation that makes the per-user solve linear in the number of items the user interacted with rather than linear in the catalog size — see Matrix Factorization §5 for the derivation).

What stage 3 solves: sparsity (the latent factors generalize from observed cells); scale (the factor matrices are tiny compared to the full similarity matrix in stage 2); implicit feedback first-class (the loss function correctly handles the preference/confidence split).

What stage 3 does not solve: cold start (the latent factors are still keyed on user and item IDs, so new IDs have no embeddings); side features (the model uses only the interaction matrix, not item content or user demographics); sequence (the model treats history as a bag).

When to stop here: if your data is sparse enough that stage 2 is unreliable, your scale exceeds 100k items, and you don’t have rich side features that need to be incorporated, stage 3 is the canonical answer for medium-to-large recommenders. Many production systems run iALS as their primary algorithm for years.

6. Stage 4 — Hybrid via LightFM, or a Two-Tower Deep Model

Beyond stage 3, the production answer is a model that consumes both interaction data and side features. Two practical paths.

6.1 LightFM — feature-driven matrix factorization

LightFM (Maciej Kula, 2015) extends matrix factorization with feature embeddings: each item is represented as a sum of feature embeddings (one embedding per feature), and similarly each user. This gives cold items reasonable embeddings on day one (because the embeddings are computed from features, not from interactions).

from lightfm import LightFM
from lightfm.data import Dataset
 
dataset = Dataset()
dataset.fit(
    users=ratings["user_id"].unique(),
    items=items["item_id"].unique(),
    item_features=items["genre"].unique(),
)
 
# Build the interaction matrix and the item-feature matrix.
interactions, _ = dataset.build_interactions(
    [(r.user_id, r.item_id) for r in ratings.itertuples()]
)
item_features = dataset.build_item_features(
    [(r.item_id, [r.genre]) for r in items.itertuples()]
)
 
# Train. WARP loss is a ranking objective good for implicit data.
model = LightFM(
    loss="warp",         # Weighted Approximate-Rank Pairwise
    no_components=64,    # latent dimension
)
model.fit(interactions, item_features=item_features, epochs=15)
 
# Recommend
import numpy as np
user_idx = 0
n_items_total = item_features.shape[0]
scores = model.predict(
    user_idx,
    np.arange(n_items_total),
    item_features=item_features,
)
top_items = np.argsort(scores)[::-1][:10]

What this code does:

  • Constructs a LightFM Dataset object that maps user IDs, item IDs, and item-feature names to internal contiguous integer indices.
  • Builds the sparse interaction matrix and the sparse item-feature matrix. The item-feature matrix has shape n_items × n_features and represents which features each item has.
  • Trains a LightFM model with the WARP (Weighted Approximate-Rank Pairwise) loss, a ranking-aware loss similar in spirit to BPR (see Matrix Factorization §7) but with a more aggressive sampling strategy that helps the model rank harder negatives correctly.
  • Predicts scores for one user against all items. Because the prediction uses item_features, even items the model never saw during training (cold items) can be scored if their features were specified in the Dataset.

LightFM is the cleanest stepping-stone from pure matrix factorization to feature-aware modelling. It is much simpler to operate than a deep two-tower model and often gives competitive results.

6.2 Two-tower deep model

For larger scale and more flexibility, a Two-Tower Retrieval Model is the production answer. The deep model can consume arbitrary user features (history sequence, demographics, context) and arbitrary item features (text embeddings, image embeddings, categorical metadata), and at inference time the item embeddings are precomputed and indexed in an Approximate Nearest Neighbour structure for fast retrieval.

A minimal PyTorch sketch — production deployments typically use TensorFlow Recommenders or a similar framework with full pipeline support:

import torch
import torch.nn as nn
import torch.nn.functional as F
 
class Tower(nn.Module):
    def __init__(self, num_categorical, num_continuous, k=64):
        super().__init__()
        self.embeds = nn.ModuleList([
            nn.Embedding(card, k) for card in num_categorical
        ])
        self.mlp = nn.Sequential(
            nn.Linear(k * len(num_categorical) + num_continuous, 128),
            nn.ReLU(),
            nn.Linear(128, k),
        )
 
    def forward(self, cat_features, cont_features):
        # cat_features: list of [batch] integer tensors
        # cont_features: [batch, num_continuous] float tensor
        embedded = [emb(c) for emb, c in zip(self.embeds, cat_features)]
        x = torch.cat(embedded + [cont_features], dim=-1)
        return F.normalize(self.mlp(x), dim=-1)   # L2-normalized output
 
user_tower = Tower(num_categorical=[n_users, ...], num_continuous=10)
item_tower = Tower(num_categorical=[n_items, ...], num_continuous=5)
 
# Training step with in-batch negatives:
def train_step(user_cats, user_conts, item_cats, item_conts):
    u = user_tower(user_cats, user_conts)   # [B, k]
    v = item_tower(item_cats, item_conts)   # [B, k]
    logits = u @ v.T                          # [B, B]
    labels = torch.arange(len(u), device=logits.device)
    loss = F.cross_entropy(logits, labels)
    return loss

This is the architectural skeleton. A production system adds: sampling-bias correction (see Two-Tower Retrieval Model §3.3); careful feature engineering for the towers; an ANN index over the item embeddings (FAISS or ScaNN); a re-ranking stage; offline evaluation pipelines; A/B testing infrastructure. The full system can be many thousands of lines.

When to step up to stage 4: if cold start is a real problem (lots of new items), if you have rich side features, if your scale demands ANN indexing, or if your accuracy requirements exceed what stage 3 can deliver.

7. Stage 5 — Sequence-Aware Recommenders

For “next item” or session-based scenarios — see Sequence-Aware Recommenders. SASRec or BERT4Rec via Transformers4Rec or a custom PyTorch implementation. This stage is optional; it pays off only if order genuinely matters in your domain.

Domains where stage 5 is worth the complexity:

  • TV series / podcasts. The next episode in a series is the highest-value recommendation.
  • Music sessions. Mood drifts within a session need to be tracked.
  • E-commerce funnels. “Just bought a phone” should shift recommendations to cases and chargers, not more phones.
  • News. Reading patterns within a session are temporal.

Domains where stage 5 is overkill:

  • Static catalogs with no clear sequence. Books, films watched once and forgotten.
  • Bag-of-items recommendations. “What might you like to add to your watchlist.”

The added complexity is significant: transformer training is much more compute-intensive than iALS; production serving requires careful sequence-length management; the model is harder to debug. Build stage 5 only when stages 0–4 demonstrably fail to capture the temporal structure of your domain.

8. Validating Each Stage — Time-Based Evaluation

At every stage, evaluate on a time-based hold-out with metrics from Recommender Evaluation Metrics. Random splits leak the future and produce misleadingly optimistic numbers.

import pandas as pd
 
# Sort by timestamp, take the chronologically last 20% as test.
ratings_sorted = ratings.sort_values("timestamp")
split_point = int(0.8 * len(ratings_sorted))
train = ratings_sorted.iloc[:split_point]
test = ratings_sorted.iloc[split_point:]
 
# For each user in the test set, the relevant items are the ones they
# interacted with in the test period.
relevant_items = test.groupby("user_id")["item_id"].apply(set).to_dict()
 
def evaluate(recommender_fn, k=10):
    """Compute Recall@K and NDCG@K averaged across users."""
    import numpy as np
    recalls = []
    ndcgs = []
    for user_id, relevant in relevant_items.items():
        if not relevant:
            continue
        recs = recommender_fn(user_id, top_n=k)   # list of recommended item_ids
        if recs is None:
            continue
        hits = [1 if iid in relevant else 0 for iid in recs]
        # Recall@K
        recalls.append(sum(hits) / min(len(relevant), k))
        # NDCG@K with binary relevance
        dcg = sum(h / np.log2(i + 2) for i, h in enumerate(hits))
        ideal_hits = [1] * min(len(relevant), k)
        idcg = sum(h / np.log2(i + 2) for i, h in enumerate(ideal_hits))
        ndcgs.append(dcg / idcg if idcg > 0 else 0)
    return np.mean(recalls), np.mean(ndcgs)

Don’t skip metrics. Don’t use random splits. Don’t compare to no baseline.

9. Decision Tree — When to Stop Climbing the Stack

flowchart TD
  Start[Need a recommender] --> S0[Stage 0:<br/>Popularity baseline]
  S0 --> Q1{Beats nothing measurable?}
  Q1 -->|No need for personalization| Stop1[Ship it]
  Q1 -->|Want personalization| S1[Stage 1:<br/>Content-based]
  S1 --> Q2{Have rich item features<br/>and modest scale?}
  Q2 -->|Yes — features capture taste| Stop2[Ship it]
  Q2 -->|Need to capture taste-based affinities| S2[Stage 2:<br/>Item-item CF]
  S2 --> Q3{Sparse data,<br/>or 10⁵+ items?}
  Q3 -->|No — n² is fine| Stop3[Ship it]
  Q3 -->|Yes — need compression| S3[Stage 3:<br/>Implicit MF / iALS]
  S3 --> Q4{Need cold-start fix<br/>or rich side features?}
  Q4 -->|No| Stop4[Ship it]
  Q4 -->|Yes| S4[Stage 4:<br/>Hybrid / Two-tower]
  S4 --> Q5{Order/sequence matters?}
  Q5 -->|No| Stop5[Ship it]
  Q5 -->|Yes — TV, music, funnel| S5[Stage 5:<br/>Sequence-aware]

What this diagram shows. A decision flow for whether each stage is worth the cost. At each branch, you either ship the current stage’s model or proceed to the next. The criteria favour stopping early — most of the conditions for advancing require evidence that the current stage is insufficient. The key insight from this diagram is that complexity is a cost; advancing to the next stage requires positive evidence, not just the desire to use a fancier algorithm. The hardest discipline in recommender engineering is resisting the leap to a deep model when the simpler stages are doing the job.

10. Pitfalls and Misconceptions

Skipping the popularity baseline. Without it, you have no idea if your fancy model is actually doing anything. Always start at stage 0; always have a baseline number to beat.

Random train/test split. Leaks the future. Always split by time. This is the single most common evaluation error.

Treating implicit feedback as ratings in matrix factorization. Plugging click-counts into Funk SVD as if they were ratings produces nonsense. Use iALS or BPR for implicit data. See Explicit vs Implicit Feedback.

Tuning by “feels right” instead of by metric. A 1% improvement in NDCG@10 is real and measurable; “the recommendations look better to me” is anecdotal and unreliable. Always tune against held-out metrics.

Deploying without an A/B test. Offline metrics correlate weakly with online lift. The only way to know if a model is genuinely better in production is to A/B test it.

Adding complexity faster than capacity. Each additional stage adds engineering work, operational cost, and serving latency. Add complexity only when the simpler stage demonstrably fails.

Ignoring data quality. A stage-3 model on dirty data loses to a stage-1 model on clean data. Spend time on the data pipeline before investing in algorithmic complexity.

Forgetting cold start. Every new user and every new item is a cold-start case. The recommender must have a defined fallback for these — usually popularity (stage 0) or content-based (stage 1) — even when the primary algorithm is more sophisticated. See Cold Start Problem.

Hyperparameter tunneling. It is tempting to optimize the hyperparameters of a stage-3 iALS model for hours rather than admit that stage-2 with proper data cleaning would have been better. Step back periodically and ask whether the right answer is to tune the current stage or to step back.

11. Open Questions

  • What stage corresponds to “good enough” for your domain? Domain-dependent. News needs sequence-awareness; e-commerce often stops at stage 3 plus content features; B2B with low data volume may stop at stage 1.
  • When does the leap from stage 3 to stage 4 actually pay off in production lift? Engineering folklore says “once you have substantial side features.” There is no clean published quantification.
  • How to know in advance which stage is right? Imperfect — usually requires building stage N and stage N+1 and A/B testing them. Some heuristics (data sparsity, catalog size, feature richness) help with the initial guess.

12. See Also

Graph View

Backlinks