DeepFM

DeepFM (Deep Factorization Machine) is a hybrid deep recommender architecture introduced by Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He (Huawei Noah’s Ark Lab) in their 2017 IJCAI paper “DeepFM: A Factorization-Machine based Neural Network for CTR Prediction”. The model combines two parallel branches sharing the same input embeddings: a Factorization Machine (FM) component that explicitly captures second-order feature interactions, and a deep neural network (DNN) component that captures higher-order non-linear interactions. The two branches’ outputs are summed and passed through a sigmoid to produce the final prediction. DeepFM is the direct successor to Wide & Deep: it replaces Wide & Deep’s manually-specified cross-feature wide branch with an FM branch that learns second-order feature interactions automatically. Like Wide & Deep, DeepFM is dominantly used for CTR (Click-Through Rate) prediction in industrial advertising and recommendation systems, and it is widely deployed in production at Huawei, Alibaba, and other major Chinese platforms.

1. Why DeepFM Improves on Wide & Deep

Wide & Deep’s wide branch requires manually specifying cross-product features (e.g., “user installed Netflix AND user shown Pandora”). This is labor-intensive and limited to feature pairs the engineer can think of. The infinite combinatorial space of possible cross-features is not actually explorable by hand.

DeepFM’s contribution is to learn the second-order cross-features automatically using the FM mechanism: every feature gets a latent embedding, and pairwise interactions are computed as dot products of embeddings. With no manual feature engineering, DeepFM captures every pairwise interaction. The deep branch handles higher-order non-linearities. The combination matches or exceeds Wide & Deep on most CTR benchmarks while removing the engineering burden.

The other contribution worth highlighting: DeepFM shares embeddings between the FM branch and the deep branch. A single embedding lookup feeds both branches, so the model has fewer total parameters than two independent models would, and training is end-to-end joint.

2. The Architecture

flowchart TD
  Input[Sparse + dense features]
  Input --> EmbLayer[Shared embedding layer]
  EmbLayer --> FM[FM Component]
  EmbLayer --> Concat
  EmbLayer --> DNN[Deep Component<br/>(MLP)]
  FM --> Add
  DNN --> Add
  Add[Sum] --> Sig[Sigmoid]
  Sig --> Pred[Predicted CTR]

What this diagram shows. The input layer takes both sparse categorical features and dense continuous features. A shared embedding layer maps each feature to a k-dimensional vector. The same embeddings are then consumed by two parallel branches: the FM component computes the linear and second-order interaction terms (Σ w_i x_i + Σ_{i<j} ⟨v_i, v_j⟩ x_i x_j); the deep component concatenates the embeddings into a long vector and passes them through a multi-layer perceptron. The two branches’ output scalars are summed and passed through a sigmoid. The key insight from this diagram is that the same embeddings serve both branches. The FM uses them for explicit second-order interactions; the DNN uses them for implicit higher-order interactions through the deep layers. This sharing is what distinguishes DeepFM from a naive ensemble.

3. Mathematical Formulation

The DeepFM prediction:

ŷ(x) = σ( y_FM(x)  +  y_DNN(x) )

where:

• y_FM(x) = w_0 + Σ_{i=1}^{d} w_i · x_i + Σ_{i=1}^{d} Σ_{j=i+1}^{d} ⟨v_i, v_j⟩ · x_i · x_j is the standard FM prediction.
• y_DNN(x) = MLP( Concat(v_1 · x_1, v_2 · x_2, ..., v_d · x_d) ) is the deep branch’s prediction; the MLP takes the concatenated active embeddings as input.
• σ is the sigmoid converting the combined logit to a probability.

The shared embeddings v_i are used in both y_FM and y_DNN. During training, gradients from both branches flow back through the same embedding parameters, encouraging the embeddings to capture both pairwise-interaction structure (used by FM) and higher-order structure (extracted by the DNN).

4. Training

Trained with stochastic gradient descent on binary cross-entropy loss:

L = − Σ_{(x, y)}  [ y · log( ŷ(x) )  +  (1 − y) · log( 1 − ŷ(x) ) ]

Standard hyperparameters: embedding dimension 4 to 32 (smaller than for pure FM because the deep branch absorbs some structure), DNN depth 2–4 hidden layers with 100–400 units each, learning rate 0.0001 to 0.001 with Adam optimizer.

The model is supported by all major DeepCTR libraries (DeepCTR, RecBole, TensorFlow Recommenders).

5. Strengths

• No manual cross-feature engineering — improvement over Wide & Deep.
• Captures both low- and high-order interactions in one model.
• Shared embeddings reduce parameter count and improve generalization.
• Strong CTR baseline in industrial settings.
• End-to-end training simplifies the engineering pipeline.

6. Weaknesses

• Pairwise FM interactions only. Higher-order explicit interactions are captured only implicitly through the DNN; xDeepFM addresses this with the Compressed Interaction Network.
• Embedding dimension is shared between branches, which couples the two components’ representational requirements.
• No native sequence handling. Sequence-aware variants exist (DIN, BST) but DeepFM by itself is bag-of-features.

7. See Also

• Wide & Deep — the predecessor DeepFM improves upon
• Factorization Machines — the FM branch’s mechanism
• xDeepFM — successor that adds the Compressed Interaction Network
• Deep & Cross Network — alternative cross-feature learning approach
• DLRM — Meta’s similar architecture
• Recommender Algorithms Catalog
• Recommender Systems