Recommender Systems MOC

Map of Content for the recommender-systems area of the vault. This MOC organizes the atomic notes into a taxonomy that lets a reader navigate by question type — what is this thing, how does it work mechanically, what categories exist, which algorithm should I pick, how do I build one, how do I evaluate it. The MOC has two layers: §2–§5 cover the concept notes (paradigms, paradigm-internal mechanisms, evaluation, practice); §6–§13 cover the algorithm notes (one per concrete algorithm, with full paper-style derivations and code examples). The full algorithm catalog with brief glosses lives in Recommender Algorithms Catalog and is the right starting point for “I need to pick an algorithm.” Each linked note is self-contained: every abbreviation is expanded on first use, every formula is walked through symbol-by-symbol, every diagram is captioned with what it shows and what insight to take from it.

1. Foundations — The Shared Vocabulary

These notes establish the concepts every other note assumes.

• Recommender Systems — the umbrella note. The problem statement, historical milestones (GroupLens 1994, Amazon item-item 2003, Netflix Prize 2006–2009, Hu/Koren/Volinsky 2008, YouTube deep recommenders 2016), the mathematical framing, the three foundational paradigms, the multi-stage architecture, and universal properties.
• Explicit vs Implicit Feedback — the data-type distinction that drives nearly every other algorithmic choice.
• Cold Start Problem — the cross-cutting concern that motivates hybrid systems and content-based methods.

2. Paradigm Categorization — The Three Top-Level Categories

• Content-Based Filtering — recommend items whose features are similar to items the user has liked.
• Collaborative Filtering — recommend based on patterns in other users’ interactions.
• Hybrid Recommender Systems — combine paradigms; Burke’s 2002 seven-strategy taxonomy.

3. Modern Industrial Architectures

• Two-Tower Retrieval Model — dominant retrieval architecture in industrial pipelines.
• Sequence-Aware Recommenders — transformer-based next-item prediction (umbrella for SASRec, BERT4Rec, etc.).

4. Evaluation

• Recommender Evaluation Metrics — RMSE, MAE, Precision/Recall@K, MAP, NDCG, MRR; offline vs online; common splits.
• Beyond-Accuracy Objectives — diversity, novelty, serendipity, coverage, fairness.

5. Practice — Hands-On Construction

• Building a Simple Recommender — staged code walkthrough from popularity baseline through hybrid two-tower.

6. Algorithms — The Concrete Implementations

The full catalog of algorithms with brief glosses lives in Recommender Algorithms Catalog. Below is the topical breakdown.

6.1 Memory-based / Neighborhood / Linear

• User-User Collaborative Filtering — original CF (GroupLens 1994).
• Item-Item Collaborative Filtering — Amazon 2003; production-grade memory-based.
• SLIM — Sparse LInear Methods (Ning & Karypis 2011); learned coefficient matrix with L1+L2.
• EASE — Embarrassingly Shallow Autoencoder (Steck 2019); closed-form-trainable linear recommender.

6.2 Matrix Factorization Variants

• Matrix Factorization — umbrella note.
• Funk SVD — Simon Funk 2006; the foundational SGD-based explicit-feedback MF.
• iALS — Hu, Koren, Volinsky 2008; implicit Alternating Least Squares.
• BPR — Rendle et al. 2009; pairwise ranking loss for implicit data.
• WARP Loss — Weston, Bengio, Usunier 2011; active negative sampling.
• SVD++ — Koren 2008; MF augmented with implicit-feedback signal.
• timeSVD++ — Koren 2009; MF with time-varying biases and factors.
• Probabilistic Matrix Factorization — Salakhutdinov & Mnih 2007; Bayesian framing.
• Non-negative Matrix Factorization — Lee & Seung 1999 (originally); non-negativity-constrained MF.
• Factorization Machines — Rendle 2010; generalizes MF to arbitrary feature interactions.

6.3 Deep Recommender Architectures

• Neural Collaborative Filtering — He et al. 2017; MLP combiner replacing dot product (later substantially critiqued).
• DSSM — Microsoft 2013; original dual-encoder for query-document ranking; ancestor of two-tower.
• YouTube Deep Recommender — Covington et al. 2016; established multi-stage retrieval-and-ranking architecture.
• Wide & Deep — Cheng et al. (Google) 2016; joint linear + deep model for memorization + generalization.
• DeepFM — Guo et al. 2017; FM + deep network with shared embeddings.
• Deep & Cross Network — Wang et al. 2017 (DCN), 2020 (DCN-V2); explicit feature crossing per layer.
• xDeepFM — Lian et al. 2018; Compressed Interaction Network for higher-order interactions.
• DLRM — Naumov et al. (Meta) 2019; Meta’s flagship deep ranker with mixed dense/sparse features.
• AutoRec — Sedhain et al. 2015; autoencoder for collaborative filtering.
• Mult-VAE — Liang et al. 2018; variational autoencoder for collaborative filtering with multinomial likelihood.

6.4 Sequence-Aware Models

• FPMC — Rendle, Freudenthaler, Schmidt-Thieme 2010; matrix factorization + first-order Markov chains.
• GRU4Rec — Hidasi et al. 2016; first widely-adopted deep sequential recommender (recurrent network).
• Caser — Tang & Wang 2018; convolutional sequence embedding.
• SASRec — Kang & McAuley 2018; causal self-attention sequential recommender (GPT-style).
• BERT4Rec — Sun et al. 2019; bidirectional self-attention with masked-item prediction (BERT-style).
• BST — Chen et al. (Alibaba) 2019; transformer over user history in a Wide & Deep ranker.
• Tiger — Rajput et al. (Google) 2024; generative retrieval with Semantic IDs.

6.5 Graph-Based Methods

• NGCF — Wang et al. 2019; first widely-cited graph-CNN-based recommender.
• LightGCN — He et al. 2020; simplified GCN that outperforms NGCF.
• PinSage — Ying et al. (Pinterest) 2018; web-scale graph CNN with random-walk sampling.

6.6 Re-Ranking and Diversity

• MMR — Carbonell & Goldstein 1998; greedy relevance + dissimilarity re-ranker.
• Determinantal Point Processes — DPPs for recommendation diversity (Chen, Zhang, Zhou 2018).

6.7 Multi-Armed Bandit / Exploration

• Multi-Armed Bandit Algorithms — umbrella covering ε-greedy, UCB1, Thompson sampling.
• LinUCB — Li et al. (Yahoo) 2010; contextual bandit with linear reward models.

6.8 Hybrid and Cold-Start Specific

• LightFM — Maciej Kula 2015; matrix factorization with feature embeddings.
• DropoutNet — Volkovs, Yu, Poutanen 2017; input dropout for cold-start conditioning.
• MeLU — Lee et al. 2019; meta-learning (MAML) for cold-start recommendation.

6.9 Approximate Nearest Neighbour Infrastructure

• FAISS — Meta’s open-source ANN library.
• ScaNN — Google’s ANN library with anisotropic product quantization.
• HNSW — Hierarchical Navigable Small World graphs (Malkov & Yashunin 2016).
• Product Quantization — Jégou, Douze, Schmid 2011; vector compression for ANN.
• Inverted File Index — IVF; coarse Voronoi-cell partitioning.
• Locality-Sensitive Hashing — LSH; hash-based ANN.

7. Cross-Reference Map

graph LR
  RS[Recommender Systems]
  CAT[Algorithms Catalog]
  EF[Explicit vs Implicit Feedback]
  CS[Cold Start Problem]
  CB[Content-Based Filtering]
  CF[Collaborative Filtering]
  HY[Hybrid Recommenders]
  EV[Evaluation Metrics]
  BA[Beyond-Accuracy]
  BL[Building a Simple Recommender]
  TT[Two-Tower Retrieval]
  SQ[Sequence-Aware]

  RS --> CB & CF & HY
  RS --> EF
  CB & CF --> HY
  CS --> HY
  EF --> CF
  RS --> EV --> BA
  RS --> CAT
  CB & CF & HY --> BL
  RS --> TT & SQ
  CAT --> RS

What this diagram shows. A directed graph of conceptual dependencies among the umbrella notes. Each algorithm note in §6 above links back to its parent paradigm (e.g., Funk SVD links to Matrix Factorization which links to Collaborative Filtering). The catalog (Recommender Algorithms Catalog) is bidirectionally connected — it lists every algorithm and links from each algorithm note. The key insight is that this is not a flat list; the algorithms inherit context from the paradigm and umbrella notes, which themselves form a hierarchy.

8. Reading Paths

• “What are recommenders?” → Recommender Systems → Content-Based Filtering → Collaborative Filtering → Hybrid Recommender Systems
• “Which algorithm should I pick?” → Recommender Algorithms Catalog → drill into specific algorithm notes
• “How do I build one?” → Building a Simple Recommender → drill into technique notes
• “How do I evaluate it?” → Recommender Evaluation Metrics → Beyond-Accuracy Objectives
• “Modern industrial pattern?” → Recommender Systems §5 → Two-Tower Retrieval Model → FAISS / ScaNN → Sequence-Aware Recommenders
• “Cold start strategy?” → Cold Start Problem → LightFM / DropoutNet / MeLU / Tiger
• “Sequential recommendation?” → Sequence-Aware Recommenders → SASRec / BERT4Rec → Tiger
• “CTR ranking?” → Wide & Deep → DeepFM → Deep & Cross Network → xDeepFM / DLRM / BST

9. Canonical Sources — Papers Worth Reading in Primary Form

The publications most heavily cited across the atomic notes:

Paper	Year	Why it matters
Resnick et al. — GroupLens	1994	Foundational automated CF.
Linden, Smith & York — Amazon Item-to-Item CF	2003	Most influential industry paper; IEEE 20-year impact award.
Funk — Netflix Update: Try This at Home	2006	Funk SVD; foundational explicit-feedback MF.
Salakhutdinov & Mnih — Probabilistic MF	2007	Bayesian framing of MF.
Koren — Factorization Meets the Neighborhood (SVD++)	2008	SVD++; bridges explicit and implicit.
Hu, Koren & Volinsky — Implicit Feedback CF (iALS)	2008	ICDM 10-Year Highest Impact; foundational implicit-feedback CF.
Koren, Bell & Volinsky — MF Techniques	2009	Canonical Netflix-Prize-era MF synthesis.
Rendle et al. — BPR	2009	Pairwise ranking loss for implicit feedback.
Koren — timeSVD++	2009	Time-varying MF; biggest single Netflix Prize gain.
Rendle et al. — FPMC	2010	First influential personalized sequential recommender.
Rendle — Factorization Machines	2010	Generalizes MF to arbitrary features.
Li et al. — LinUCB	2010	Contextual bandit for personalized news.
Jégou, Douze & Schmid — Product Quantization	2011	Vector compression for ANN.
Ning & Karypis — SLIM	2011	Sparse linear methods for top-N.
Burke — Hybrid Recommender Systems	2002	Seven-strategy hybridization taxonomy.
Huang et al. — DSSM	2013	Dual-encoder for web search; ancestor of two-tower.
Sedhain et al. — AutoRec	2015	Autoencoder for CF.
Hidasi et al. — GRU4Rec	2016	First widely-adopted deep sequential recommender.
Cheng et al. — Wide & Deep	2016	Memorization + generalization architecture.
Covington et al. — YouTube Deep	2016	Multi-stage retrieval/ranking architecture standard.
Malkov & Yashunin — HNSW	2016	Graph-based ANN with logarithmic complexity.
Volkovs et al. — DropoutNet	2017	Input dropout for cold-start.
Wang et al. — DCN	2017	Explicit feature crossing per layer.
Guo et al. — DeepFM	2017	FM + deep network with shared embeddings.
He et al. — NCF	2017	MLP combiner replacing dot product.
Kaminskas & Bridge — Beyond-Accuracy	2017	Diversity, serendipity, novelty, coverage synthesis.
Liang et al. — Mult-VAE	2018	Variational autoencoder for CF.
Lian et al. — xDeepFM	2018	Compressed Interaction Network.
Tang & Wang — Caser	2018	Convolutional sequence embedding.
Kang & McAuley — SASRec	2018	Causal self-attention sequential recommender.
Ying et al. — PinSage	2018	Web-scale graph CNN with random walks.
Chen, Zhang & Zhou — DPP MAP Inference	2018	Fast greedy MAP for DPP-based diversity.
Sun et al. — BERT4Rec	2019	Bidirectional masked-item-prediction recommender.
Naumov et al. — DLRM	2019	Meta’s flagship deep ranker.
Wang et al. — NGCF	2019	First widely-cited GCN-based recommender.
Yi et al. — Sampling Bias Correction	2019	Essential correction for two-tower in-batch negatives.
Lee et al. — MeLU	2019	Meta-learning (MAML) for cold-start.
Steck — EASE	2019	Closed-form-trainable linear recommender.
Chen et al. (Alibaba) — BST	2019	Transformer over behavior sequence in CTR ranker.
He et al. — LightGCN	2020	Simplified GCN-CF.
Rendle et al. — NCF vs MF Revisited	2020	Decisive critique of NCF claim.
Krichene & Rendle — Sampled Metrics	2020	Critique of sampled-metric evaluations.
Guo et al. — ScaNN AVQ	2020	Anisotropic product quantization for inner-product search.
Wang et al. — DCN-V2	2020	Improved DCN with full weight matrices.
Petrov & Macdonald — SASRec vs BERT4Rec	2023	Reckoning that revised consensus.
Rajput et al. — TIGER	2024	Generative recommender with Semantic IDs.

10. Open Threads — Topics Identified But Not Yet Written Up

• Knowledge-based and context-aware recommenders. Covered briefly in Recommender Systems §4.4.
• Causal recommendation and off-policy evaluation. Frontier; touched in Recommender Evaluation Metrics.
• LLM-based recommenders beyond TIGER. Production viability remains open.
• Multi-stakeholder recommenders for two-sided marketplaces.
• Privacy-preserving recommendation. Federated learning, on-device, differential privacy.
• Production engineering: feature stores, online serving, A/B testing infrastructure.
• Reinforcement-learning recommenders beyond contextual bandits.
• Modern transformer rankers (Transformers4Rec, RecBole-DCN-style) beyond what’s covered.