Wide & deep learning for recommender systems

A recommender system with a wide & deep model (WDL).

Meta Info

Presented in arxiv:1606.07792. (Accepted in DLRS 2016)

Authors: Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, Hemal Shah (Google).

Understanding the paper

Background

• A recommender system

  • A search ranking system

  • Input: a set of user and contextual information

  • Output: a ranked list of items

  • Workflow

    • Retrieval

      • The retrieval system returns a short list of items that best match the query using various signals, usually a combination of machine-learned models and human-defined rules.

    • Rank (this work focuses on the ranking model)

      • Rank all items by their scores.

Wide & Deep model structure

• Wide component

  • Generalized linear model

  • Cross-product transformation

• Deep component

  • Feed-forward NN

  • High-dimensional categorical features are converted into a low-dimensional and dense real-valued vector, often referred to as an embedding vector.

  • The dimensionality of the embeddings is usually on the order of O(10) to O(100).

Wide & Deep model structure for apps recommendation.

Recommendation pipeline overview

• Model Training

  • Warm-start: initialize a new model with the embeddings and the linear model weights from the previous model

  • Dry-run the model before loading the models into the servers

• Model Serving

  • Each request includes a set of item candidates and user features

  • Use multithreading parallelism to run smaller batches in parallel, instead of scoring all candidate items in a single-batch inference

Apps recommendation pipeline overview.