PAPERS-11-006 · Paper — DeepFM

The paper

DeepFM, introduced by Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He in 2017 at the International Conference on World Wide Web, is considered a landmark because it unified low‑order and high‑order feature learning in a single, end‑to‑end model for click‑through‑rate prediction.

Before this paper

Before DeepFM, the popular Wide & Deep model still required hand‑engineered feature crosses for its wide component. This reliance on expert‑crafted interactions limited scalability and often missed useful combinations of features. Practitioners had to invest significant effort to design these crosses, and the model could not automatically discover all useful low‑order interactions.

The key idea

The key insight of DeepFM is to combine a factorization machine with a deep neural network while sharing the same feature embeddings. This lets the model learn both pairwise (low‑order) and high‑order interactions automatically, removing the need for manual feature crossing. By using a single set of embeddings, the system remains simple and can be trained end‑to‑end.

How it works

DeepFM first maps each raw input feature into a dense embedding vector. These embeddings are fed simultaneously into two branches. The first branch is a factorization‑machine component that computes pairwise interactions by taking inner products of embedding pairs, capturing low‑order relationships. The second branch passes the same embeddings through a deep neural network (multiple hidden layers) to model high‑order, nonlinear interactions. Both branches share the embeddings, so gradients from both parts update the same vectors during training, enabling joint learning without any hand‑crafted feature crosses.

See it — FM and deep, shared embeddings

flowchart LR
  A["Shared embeddings"] --> B["FM part:<br/>low-order"]
  A --> C["Deep part:<br/>high-order"]
  B --> D["Combine"]
  C --> D

What they showed

The authors showed that DeepFM matched or beat Wide & Deep on click‑through‑rate prediction tasks, achieving comparable or superior performance without any hand‑crafted feature crosses.

What it changed

After its release, DeepFM quickly became a popular click‑prediction architecture in industry, influencing many production systems for ads and recommendation. Its ability to learn interactions automatically reduced engineering effort and improved model performance, leading to widespread adoption in real‑world ranking pipelines. The approach inspired a family of "DeepFM‑style" models that are now standard components in many large‑scale recommendation platforms. Next: DLRM.

The one-line takeaway

DeepFM proves that sharing embeddings between a factorization machine and a deep network can automatically capture both low‑ and high‑order feature interactions without manual feature engineering.

Read it / meet it

Reading the original DeepFM paper gives a clear view of how simple architectural choices can transform click‑prediction systems. Today, you can see its legacy in the models that rank ads and feed items across major online platforms.