vault

Caser

5 min read

Caser

Caser (Convolutional Sequence Embedding Recommendation) is a sequential recommendation model introduced by Jiaxi Tang and Ke Wang in their 2018 WSDM paper “Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding”. The model takes a fundamentally different approach from the recurrent (GRU4Rec) and self-attention (SASRec, BERT4Rec) sequential recommenders: it treats the user’s recent item sequence as a 2D image in time × latent-dimension space and applies convolutional filters — both horizontal (time-axis) and vertical (latent-axis) — to capture sequential patterns as local features. The horizontal filters capture point-level and union-level patterns (e.g., “user always buys A and B together”); the vertical filters capture skip behaviors (e.g., “user buys A then waits then buys C, with B sometimes interspersed”). Caser combines these convolutional sequence embeddings with a personalized user embedding via a fully-connected layer to produce next-item predictions. Caser was state-of-the-art for top-N sequential recommendation in 2018 and remains a strong baseline; it has been somewhat displaced by transformer-based methods but offers an interesting alternative architecture worth knowing for cases where convolutional inductive biases are appropriate.

1. The Convolutional Approach

FPMC models first-order Markov transitions; GRU4Rec models recurrent dependencies through gated recurrence; SASRec uses self-attention. Caser uses convolutional filters — the same operation used in image processing and convolutional language models.

The setup: take the user’s most recent L items (the recent sequence), look up each one’s embedding, and arrange them as an L × d matrix where d is the embedding dimension. This matrix is conceptually a 2D “image” — the time axis is one direction; the latent-dimension axis is the other. Convolutional filters of various shapes are then applied to this image to extract sequential patterns.

The two key filter types:

Horizontal filters of shape h × d (where h is the number of consecutive time steps) slide along the time axis. They span all d latent dimensions but only h time steps. A horizontal filter learns a pattern like “these items appearing in this time-relative configuration matter together.” With multiple horizontal filters of different h values (e.g., h ∈ {2, 3, 4}), the model captures patterns of various time-spans.

Vertical filters of shape L × 1 slide along the latent-dimension axis. They span the full L time steps but only one latent dimension. A vertical filter learns a pattern in the temporal evolution of a single latent feature, capturing skip-like patterns where features matter at different time positions.

2. The Architecture

flowchart LR
  Hist[Recent L items<br/>i_{t-L+1}, ..., i_t] --> Emb[Embedding lookup]
  Emb --> Mat[L × d embedding matrix<br/>("image")]
  Mat --> HF[Horizontal filters<br/>(h × d shapes)]
  Mat --> VF[Vertical filters<br/>(L × 1 shapes)]
  HF --> HM[Max-pooled horizontal<br/>features]
  VF --> VM[Vertical features]
  User[User u] --> UE[User embedding]
  HM --> Concat
  VM --> Concat
  UE --> Concat
  Concat[Concatenate] --> FC[Fully-connected<br/>+ softmax over items]
  FC --> Pred[Next item probabilities]

What this diagram shows. The user’s recent L items are embedded into an L × d matrix (the “image”). Two parallel branches of convolutional filters operate on this image: horizontal filters (capturing local time patterns across multiple items) and vertical filters (capturing patterns within single time-positions across latent dimensions). The horizontal filters are typically followed by max-pooling to reduce dimensionality. A separate user embedding captures the user’s general preferences. All three feature sources — horizontal pattern features, vertical pattern features, and the user embedding — are concatenated and passed through a fully-connected layer with softmax to produce next-item probabilities. The key insight from this diagram is that the convolutional inductive bias differs from the recurrent and attention inductive biases — convolutions learn local translation-invariant patterns. This may be appropriate for short sequences with clear motifs and less appropriate for long sequences with long-range dependencies (where attention is better).

3. Training

Trained with binary cross-entropy on (positive, negative) item pairs sampled from the user’s history and from random negatives:

L = − Σ ( log σ(ŷ_pos)  +  log(1 − σ(ŷ_neg)) )

Adam optimizer; embedding dimension typically 50–100; horizontal filter sizes h ∈ {2, 3, 4}; number of filters per size 16–64; user embedding dimension same as item embedding.

The PyTorch reference implementation provides standard hyperparameter defaults.

4. Captured Patterns

The paper distinguishes three sequential pattern types Caser captures:

Point-level patterns — single items in the recent history matter. A horizontal filter with h = 1 captures these.

Union-level patterns — combinations of items that appear together (regardless of exact time position) matter. Horizontal filters with h > 1 capture these.

Skip behaviors — items at non-adjacent time positions matter together; the model should recognize that intervening items might be irrelevant. Vertical filters capture these by spanning the full time axis.

Together, the three pattern types cover most sequential recommendation needs. The trade-off versus SASRec is that convolutional filters have local receptive fields, while self-attention is fully-connected (every position attends to every other). For longer sequences, attention is more expressive; for shorter sessions with clear local motifs, convolutions may be more parameter-efficient.

5. When to Use Caser

Caser is appropriate when:

  • The relevant context is short (recent 5–20 items) and patterns are local.
  • You want a simpler architecture than transformers.
  • You need both general user preferences and sequential context in one model.

Do not use Caser when:

  • Long sequences (100+ items) matter — self-attention is more appropriate.
  • You need cold-item handling beyond what ID-based models offer.

6. Strengths

  • Captures multiple pattern types (point, union, skip) through different filter shapes.
  • Personalized + sequential — user embedding plus session features.
  • Simpler than transformers for short sequences.
  • Strong empirical results in 2018; remains a competitive baseline.

7. Weaknesses

  • Local receptive field — long-range dependencies require many layers or large filters.
  • Bounded recent context L is a hyperparameter that must be set in advance.
  • Outperformed by transformer-based methods on most modern benchmarks.

8. See Also

Graph View

Backlinks