vault

Neural Collaborative Filtering

20 min read

Neural Collaborative Filtering

Neural Collaborative Filtering (NCF) is a recommender architecture introduced by Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu and Tat-Seng Chua in the 2017 WWW (World Wide Web) conference paper “Neural Collaborative Filtering”. The framework’s central proposal is to replace the inner-product step of Matrix Factorization with a multi-layer perceptron (MLP, a feed-forward neural network with one or more hidden layers). User and item embeddings are learned in essentially the same way as in matrix factorization; what changes is how the two embeddings are combined into a final score. The argument was that the dot product is linear in the latent factors and therefore structurally limited in what user-item interactions it can express, and that an MLP — being a universal function approximator — could in principle capture any interaction function. The paper became enormously influential, with thousands of citations, and it kicked off a line of deep collaborative filtering research that dominated recommender venues for several years. The story turned out to have a twist: in 2020, Rendle, Krichene, Zhang and Anderson published “Neural Collaborative Filtering vs. Matrix Factorization Revisited” at RecSys 2020, showing that a properly-tuned dot-product matrix factorization matches or beats the NCF architecture on the same benchmarks. The Rendle critique substantially deflated the specific NCF claim — that the MLP combiner is necessary — but did not undo the broader move toward deep models, which proceeded for other reasons.

1. Origin and Motivation

The 2017 NCF paper opened with a critique of matrix factorization’s expressive power. Standard MF predicts:

r̂_{u,i} = x_uᵀ · y_i = Σ_{f=1}^{k}  x_{u,f} · y_{i,f}

where x_u, y_i ∈ ℝ^k are the latent vectors and the dot product is a sum of element-wise products. The dot product is linear in each factor — meaning the contribution of factor f to the predicted score is x_{u,f} · y_{i,f}, with no interaction with other factors. The model can capture additive interactions (“user with high factor-1 likes items with high factor-1”) but cannot capture conditional interactions (“user likes action movies, but only if they are under two hours, and only on weekends”).

He et al. argued this was a fundamental limitation that prevented matrix factorization from modeling complex user-item interactions, and they proposed replacing the dot product with a learned function:

r̂_{u,i} = f(x_u, y_i ; Θ)

where f is a multi-layer perceptron parameterized by weights Θ learned from data. Because an MLP is a universal function approximator (Hornik 1991), it can in principle represent any interaction function, removing the linearity constraint. This was the paper’s central architectural innovation.

The empirical evaluation in the paper claimed substantial improvements on two datasets — MovieLens 1M and a Pinterest dataset — over baseline methods including Item-Pop (popularity), ItemKNN (Item-Item Collaborative Filtering), BPR (Matrix Factorization with the BPR loss), and eALS (an improved variant of iALS). The largest gains came from adding the MLP path on top of a generalized matrix-factorization baseline.

The paper was published at WWW 2017, became one of the most-cited recommender-systems papers of the late 2010s, and effectively launched the era of deep collaborative filtering. For several years, dozens of follow-up papers proposed variants of the NCF framework with different architectures (convolutional layers, attention, residual connections).

2. The NCF Framework — Three Architectures

The 2017 paper specified three concrete architectures within the NCF framework, used in different combinations.

flowchart TD
  uID[User ID one-hot] --> uEmb[User Embedding x_u ∈ ℝ^k]
  iID[Item ID one-hot] --> iEmb[Item Embedding y_i ∈ ℝ^k]
  uEmb --> Branch
  iEmb --> Branch
  Branch[Combiner choice]
  Branch --> GMF[GMF: element-wise product → linear layer]
  Branch --> MLP[MLP: concatenate → hidden layers → output]
  Branch --> NeuMF[NeuMF: GMF + MLP, concatenate outputs]
  GMF --> Score
  MLP --> Score
  NeuMF --> Score[σ(z): predicted preference]

What this diagram shows. The leftmost two nodes are one-hot encodings of the user ID and item ID — sparse vectors with a single 1 indicating which user or item this is. These are passed through embedding lookups that produce dense k-dimensional vectors x_u and y_i. The middle “combiner choice” node represents the architectural decision: what function combines x_u and y_i into a score. The paper proposes three options. GMF (Generalized Matrix Factorization) takes the element-wise product x_u ⊙ y_i and feeds it through a learned linear layer. With the linear layer’s weights initialized to all-ones, GMF recovers the standard MF dot product exactly; with arbitrary learned weights, it generalizes MF by allowing each dimension’s contribution to be re-weighted. MLP concatenates [x_u ; y_i] into a 2k-dimensional vector and passes it through one or more hidden layers with non-linear activations (typically ReLU). NeuMF (Neural Matrix Factorization) is an ensemble that combines GMF and MLP: each branch produces an output vector; these are concatenated and passed through a final layer to produce the score. The output is passed through a sigmoid σ to produce a probability in (0, 1), interpreted as predicted preference. The key insight from this diagram is that the embedding stage is identical to matrix factorization — the architectural innovation is entirely in the combiner. The embeddings still capture the same kind of latent structure; the question is whether the combiner can extract more from them than the simple dot product can.

2.1 GMF — Generalized Matrix Factorization

ŷ_{GMF} = a_out ( wᵀ · ( x_u ⊙ y_i ) )

where:

  • x_u ⊙ y_i ∈ ℝ^k is the element-wise (Hadamard) product of the user and item embeddings.
  • w ∈ ℝ^k is a learned weight vector.
  • a_out is the output activation (typically sigmoid for binary classification, identity for regression).
  • Setting w = (1, 1, ..., 1) and a_out = identity recovers the standard MF dot product exactly.

GMF is therefore a strict generalization of dot-product matrix factorization. With a learned w, it can re-weight the importance of each factor.

2.2 MLP — Multi-Layer Perceptron

z_0 = [ x_u ; y_i ]                # concatenation, shape 2k
z_l = a_l ( W_l · z_{l-1} + b_l )  # hidden layer l
ŷ_{MLP} = a_out ( wᵀ · z_L )       # final scalar score

where:

  • [ x_u ; y_i ] is the concatenation of the two embedding vectors.
  • W_l and b_l are the weights and biases of hidden layer l.
  • a_l is the activation function (typically ReLU: ReLU(x) = max(0, x)).
  • L is the number of hidden layers (the paper used 3 in many experiments).
  • w is the final output weight vector and a_out is the sigmoid for binary classification.

The MLP can in principle learn any function of (x_u, y_i), including non-linear interactions the dot product cannot.

2.3 NeuMF — Neural Matrix Factorization

The flagship architecture of the paper. Combines GMF and MLP with separate embedding layers:

z_GMF = x_u^{(GMF)} ⊙ y_i^{(GMF)}                              # GMF branch
z_MLP = MLP( [ x_u^{(MLP)} ; y_i^{(MLP)} ] )                    # MLP branch
ŷ_{NeuMF} = σ( h^T · [ z_GMF ; z_MLP ] )                        # combine and predict

where:

  • The GMF branch and MLP branch each have their own user and item embedding tables — x_u^{(GMF)} and x_u^{(MLP)} are different vectors for the same user.
  • The two branches’ outputs are concatenated into [ z_GMF ; z_MLP ] and passed through a final linear layer with weight vector h.
  • σ(z) = 1 / (1 + e^{−z}) is the sigmoid activation.

The two-branch design lets GMF capture the linear interaction structure and the MLP capture non-linear residuals, with the final layer learning how much weight to give each branch.

Pretraining matters for NeuMF. Because the NeuMF objective is non-convex, the original paper found that random initialization led to poor local optima, so it specified a two-stage training recipe (He et al. 2017, §3.4.1): first train the GMF and MLP models separately to convergence (using the Adam optimizer), then initialize NeuMF with those pre-trained weights. The only blend point is the final output layer, whose weight is set to h ← [α·h_GMF ; (1−α)·h_MLP] — a convex combination of the two pre-trained output heads, with α = 0.5 so each branch contributes equally at the start. One subtlety the paper calls out: after pretraining, NeuMF is fine-tuned with vanilla SGD, not Adam, because Adam relies on per-parameter momentum estimates that were not carried over from the separate pretraining runs, so resuming with Adam would be unsound. The paper reports pretraining gave NeuMF roughly a 1–2% relative HR@10 / NDCG@10 lift over training the ensemble from scratch — a real but modest gain that, in light of the Rendle critique (§4.1), is dwarfed by the gap to a tuned dot product.

3. The Loss Function

The paper assumes implicit feedback — observations are binary p_{u,i} ∈ {0, 1} indicating whether the user has interacted with the item. The loss is binary cross-entropy with negative sampling:

L = − Σ_{(u, i) ∈ Y⁺}  log σ( ŷ_{u,i} )  −  Σ_{(u, j) ∈ Y⁻}  log( 1 − σ( ŷ_{u,j} ) )

where:

  • Y⁺ is the set of observed positive (user, item) interactions.
  • Y⁻ is a set of sampled negative (user, item) pairs — for each observed positive (u, i), draw k items the user has not interacted with and treat them as negatives. Typical k: 4 to 8.
  • ŷ_{u,i} is the model’s score for (u, i), before the sigmoid.
  • σ( ŷ_{u,i} ) is the predicted probability of positive interaction.
  • The first term penalizes low predicted probability for observed positives; the second term penalizes high predicted probability for sampled negatives.

This is the standard binary cross-entropy loss applied to recommendation, with the negatives sampled rather than enumerated (because the catalog is too large to enumerate every non-interaction). See Explicit vs Implicit Feedback §6 for the treatment of negative sampling.

The paper trains with mini-batch Adam on the loss above (for the GMF and MLP models, and for NeuMF from scratch; pretrained NeuMF is fine-tuned with vanilla SGD — see §2.3). The implementation was built on Keras (He et al. 2017, §4.1). The verified hyperparameter grid in the paper: the number of “predictive factors” (the embedding/output dimension) was swept over {8, 16, 32, 64}; the default MLP used 3 hidden layers in a halving tower (e.g., for predictive factors 8, the neural-CF layers are 32 → 16 → 8, with the 2k-wide concatenation at the bottom); batch size searched over {128, 256, 512, 1024}; learning rate over {0.0001, 0.0005, 0.001, 0.005}; 4 negative samples per positive for the from-scratch runs. A sweep over MLP depth (0 to 4 hidden layers, Tables 3–4 in the paper) showed deeper towers helped within the original paper’s own evaluation — a result the Rendle critique later reframed (§4.1).

4. The Empirical Claim — and the Subsequent Pushback

The paper’s headline claim was that NeuMF outperformed all baselines tested — Item-Pop, ItemKNN, BPR, eALS — on two benchmark datasets (MovieLens 1M and a Pinterest dataset). The reported metrics were Hit Rate at 10 (HR@10) and Normalized Discounted Cumulative Gain at 10 (NDCG@10), evaluated under a leave-one-out protocol: each user’s most recent interaction is held out as the test target, and rather than rank the full catalog (too slow), the model ranks that one held-out item against 100 randomly sampled items the user has not interacted with, then HR@10 / NDCG@10 are computed over that list of 101 (He et al. 2017, §4.1). This sampled-ranking protocol is itself now considered problematic for absolute comparisons — Krichene & Rendle’s “On Sampled Metrics for Item Recommendation” (KDD 2020) showed sampled metrics can be inconsistent with full-catalog metrics — but Rendle et al. (§4.1 below) deliberately reused the identical protocol so the comparison to NCF stays apples-to-apples. See Recommender Evaluation Metrics for definitions of these metrics.

For three years, NCF’s claimed superiority over matrix factorization was widely accepted as a settled result, and the field moved toward deep architectures.

4.1 The Rendle 2020 Critique

In 2020, Steffen Rendle, Walid Krichene, Li Zhang and John Anderson (all at Google Research) published “Neural Collaborative Filtering vs. Matrix Factorization Revisited” at RecSys 2020. The abstract states the thesis plainly: “with a proper hyperparameter selection, a simple dot product substantially outperforms the proposed learned similarities,” and “while a MLP can in theory approximate any function, we show that it is non-trivial to learn a dot product with an MLP.” Three threads carry that thesis, and it is worth being precise about each — the third one is routinely misremembered.

  1. The original comparison was unfair, and a properly-configured dot-product baseline wins. Rendle et al. re-ran the exact NCF evaluation — the same MovieLens 1M and Pinterest splits, the same leave-one-out protocol, the same HR@10 / NDCG@10 metrics, the same log loss with negative sampling — changing only the similarity function. A plain matrix-factorization baseline (a dot product with bias terms, trained with the same loss and sampling) beat NeuMF and the MLP on all datasets, all metrics, and all embedding dimensions but one (§3.3). The concrete numbers from their Table 1 (embedding dimension d = 192): on MovieLens, MF reached HR@10 = 0.7294 / NDCG@10 = 0.4523 versus NeuMF’s 0.7093 / 0.4349; on Pinterest, MF reached 0.8895 / 0.5794 versus NeuMF’s 0.8777 / 0.5576. They also noted (citing Dacrema et al.’s reproducibility meta-study) that the NeuMF and MLP numbers reported in the original NCF paper were cherry-picked in the sense that the metrics were taken at the iteration that looked best on the test set, which over-estimates true performance — meaning the real gap is even larger than the published tables suggest.

  2. GMF and NeuMF do not vindicate the learned combiner either. The paper checked NCF’s weaker claims too. NeuMF (the GMF + MLP ensemble) showed “only a minor improvement over MLP and overall a much worse quality than MF,” so the experiments “do not support the claim that a dot product model can be enhanced by feeding some part of its embeddings through an MLP.” They further showed that GMF’s learned per-dimension weight vector w adds no expressive power — it can be folded into the embeddings (w ⊙ p absorbs into p) — and actually breaks L2 regularization (the loss becomes invariant to a rescaling P/a, Q/a, a²w, so regularization no longer constrains the solution), explaining why GMF underperformed and why simply adding parameters to a simple model “is not always a good idea.”

  3. The real point is inductive bias, not raw expressive capacity. This is the thread the note’s earlier draft got wrong, and that interview answers routinely garble. Rendle et al. do not argue “the dot product is not a limitation because the embeddings can encode anything.” They concede the textbook fact that an MLP is a universal approximator — and then argue that universality is the wrong lens. The dot product is a useful structural prior for a similarity function, and the MLP’s far larger hypothesis class means it needs much more capacity and data to learn that same structure from scratch. Their dedicated experiment (§4, “Learning a Dot Product with MLP is Hard”) trains an MLP to imitate a known dot product over Gaussian embeddings and finds the number of training samples needed to reach a target error grows roughly as O((d/ε)^α) with 1 ≤ α ≤ 2 (they cite a theoretical bound of O(d⁴/ε²) steps for learning a degree-2 polynomial). The conclusion (§1): “While MLPs can approximate any continuous function, their inductive bias might not be well suited for a similarity measure. Unless the dataset is large or the embedding dimension is very small, a dot product is likely a better choice.” So the dot product’s “linearity,” far from being a defect, is a helpful bias — exactly the inverse of the 2017 framing.

The Rendle paper recommended that any future deep-recommender claim be benchmarked against a properly-tuned dot-product MF baseline. The recommendation was widely adopted, and a parallel reproducibility wave (notably Dacrema, Cremonesi & Jannach’s “Are We Really Making Much Progress?” RecSys 2019) found many “deep beats X” results evaporated under fair tuning. See the paper’s code repository for reproductions.

It is important to be precise about what this critique does and does not establish. It deflates one specific architectural claim — that a learned MLP combiner beats the dot product. It does not show that deep learning is useless for recommendation; deep models still dominate production, but for reasons orthogonal to the NCF combiner (richer input features, sequence modeling, multi-modality — see §5). The honest one-line summary for an interview: “NCF’s learned-similarity claim did not replicate; a well-tuned dot product matches or beats it, and the MLP’s universal-approximation property is beside the point because its inductive bias for similarity is worse — but deep models still won on the input side, not the combiner side.”

5. Why the Field Still Uses Deep Models

The Rendle critique might suggest deep recommenders are not worth the complexity. This conclusion would be wrong. Deep models continue to dominate production for several reasons that are not the original NCF reason:

Side features. Pure dot-product MF cannot easily incorporate item content features, user demographics, or context. Deep models concatenate side features naturally into the embedding step (or as a separate input branch), allowing collaborative and content signals to be combined in one model. This is the feature-combination hybrid pattern at scale and is the dominant industrial design.

Sequential context. Modeling user behaviour as an ordered sequence requires recurrent or attention-based architectures that have no MF analogue. See Sequence-Aware Recommenders for the transformer-based families that have displaced MF for “next-item” recommendation tasks.

Multi-modal items. Items with image, text, audio, and structured-metadata features are most naturally handled by deep models that consume each modality through a dedicated sub-network. MF can use precomputed embeddings as input features but cannot learn the embeddings end-to-end.

Industrial retrieval. The Two-Tower Retrieval Model uses deep embeddings combined by a dot product. So in a precise sense it is MF-like (the combiner is a dot product) but with a deep encoder for each input. Two-tower models are the dominant retrieval architecture at YouTube, TikTok, Spotify and most large-scale platforms.

The pattern: deep models won because of the input side (richer features, sequential context, multimodal) rather than the combiner side. NCF’s specific architectural claim — that the combiner needed to be deeper — was the part that did not survive scrutiny.

6. Strengths

Flexible architecture. Easy to extend with side information, attention mechanisms, sequential context, or multi-modal inputs. The deep-learning toolkit transfers directly.

Captures non-linear interactions in principle. The MLP path can in theory model preferences that the dot product cannot. Whether this matters in practice is the Rendle question; in some specific cases (extreme conditional preferences) it may.

Implicit feedback first-class. The framework was designed around implicit binary signals from the start. The cross-entropy loss with negative sampling is well-suited to clicks, watches, and other behavioural data.

Easy to combine with other deep components. NCF blocks slot into larger architectures naturally — for instance, as the ranking head of a multi-stage system whose retrieval comes from a two-tower model.

7. Weaknesses

Easy to outperform with a well-tuned matrix factorization. As Rendle 2020 demonstrated, the central architectural claim of NCF does not hold up when the MF baseline is tuned competently. For pure rating/ranking accuracy on standard benchmarks, MF with proper regularization is often as good or better.

More hyperparameters. Embedding dimension, MLP depth, MLP width per layer, dropout rate, learning rate, number of negatives per positive, batch size — substantially more knobs than MF (which has essentially k and λ).

Heavier inference. Each (user, item) candidate scoring requires an MLP forward pass. For retrieval (scoring millions of candidates per request) this is structurally infeasible, which is why two-tower models with dot-product combiners dominate the retrieval stage even when deep models are used elsewhere.

Cold start unchanged. Embeddings are learned per ID, so new IDs have no embeddings. NCF inherits the Cold Start Problem from matrix factorization without solving it. Modern variants that use feature inputs (rather than ID one-hots) address this, but the original NCF architecture does not.

Less interpretable. Like all deep models, the learned representations are opaque. Diagnosing a bad recommendation is harder than in MF (where the latent factors can sometimes be inspected) or memory-based CF (where the contributing co-occurrences are concrete).

8. Pitfalls and Misconceptions

“NCF is the deep version of MF.” It is one deep version. Two-tower models with dot-product combiners are also deep versions of MF and are far more useful in production. Conflating “deep” with “NCF” misses most of the modern recommender landscape.

“Bigger MLP → better.” Diminishing returns kick in quickly. Two or three hidden layers with reasonable width is usually enough; deeper models often overfit on the typical training-data sizes available for collaborative filtering.

Comparing to a poorly-tuned MF baseline. The Rendle paper exists precisely because of this. Always tune the simple model before comparing a complex one to it. The Rendle code repository provides reference implementations for fair comparison.

Using NCF for retrieval. Per-pair MLP scoring does not index. For retrieval at scale, you need a model whose scoring decomposes into a dot product (or other inner-product-like operation) so the items can be precomputed and looked up via Approximate Nearest Neighbor search. NCF is appropriate for ranking, not retrieval.

Forgetting the negative sampling correction. Without a sampling-bias correction, in-batch or popularity-sampled negatives bias the model toward under-recommending popular items. The correction is the same as in two-tower training; see Two-Tower Retrieval Model §3.

Treating NeuMF’s two embedding tables as equivalent to MF’s one. NeuMF gives separate embeddings to the GMF and MLP branches, doubling the embedding parameters. This is a real cost in storage and training time and should be weighed against the marginal accuracy gains.

9. Open Questions

  • Why didn’t NCF age better? Part of the answer is the Rendle critique. Part is that the field moved on to architectures (two-tower, transformers, sequence-aware models) that solve other bottlenecks (retrieval scale, sequence modeling). NCF’s architectural claim — that the combiner needs to be deep — was a specific narrow claim that turned out not to hold.
  • Are there specific user-item interaction patterns where the MLP combiner genuinely beats the dot product? Theoretically yes (the dot product cannot model conditional preferences); empirically the gap is small and may be domain-dependent.
  • End-to-end deep recommendation that subsumes NCF. Modern systems combine many ideas — feature combination, deep embedding, dot-product retrieval, deep ranking — into pipelines where NCF’s specific block is one of many design choices. Whether NCF survives as a distinct named architecture in the long run is an open question.

10. See Also

Graph View

Backlinks