RecMind

Large Language Model Powered Agent For Recommendation


1,2 Yancheng Wang, 1 Ziyan Jiang, 1 Zheng Chen, 1 Fan Yang, 1 Yingxue Zhou,
1 Eunah Cho, 1 Xing Fan, 1 Xiaojiang Huang, 1 Yanbin Lu, 2 Yingzhen Yang
1Amazon Alexa AI, 2Arizona State University
arXiv Code 🤗 Dataset Twitter

Abstract

While the recommendation system (RS) has advanced significantly through deep learning, current RS approaches usually train and finetune models on task-specific datasets, limiting their generalizability to new recommendation tasks and their ability to leverage external knowledge due to model scale and data size constraints. Thus, we designed an LLM powered autonomous recommender agent, RecMind, which is capable of leveraging external knowledge, utilizing tools with careful planning to provide zero-shot personalized recommendations.

  1. RecMind Framework. We introduce RecMind, the first LLM-powered agent designed for general recommendation purposes, which operates without the need for fine-tuning for domain adaptation across datasets or tasks.
  2. Planning: Self-Inspiring. We incorporate a novel self-inspiring (SI) planning technique in RecMind. This technique integrates multiple reasoning paths and offers an empirical improvement over currently popular methods, such as CoT and ToT.
  3. Experiments: Effective and Generalizable. We evaluate the effectiveness and generalizability of RecMind across five recommendation tasks and two datasets. Extensive experiments and analyses demonstrate that RecMind outperforms state-of-the-art (SOTA) LLM-based baselines that do not involve any fine-tuning and achieves competitive performance with a fully pre-trained expert recommendation model such as P5.

RecMind Framework

MY ALT TEXT

Here is an overview of our proposed RecMind architecture. It comprises four major components: "RecMind" is built based on ChatGPT API, "Tools" support various API call to retrieve knowledge from "Memory" component, "Planning" component is in charge of thoughts generation.

Planning: Self-Inspiring

MY ALT TEXT

Planning helps LLM Agents decompose tasks into smaller, manageable subgoals for handling complex tasks.

MY ALT TEXT

Comparisons of rating prediction by RecMind-ToT (left) and RecMind-SI (right). After searching for the product category of the item in Step 2, RecMind-ToT first generates thought 3 (1) to retrieve the rating of a similar item. After being evaluated by the voting-based evaluator, RecMind-ToT prunes option 3 (1) and proposes another thought 3 (2) to retrieve the average rating of the item and then makes the prediction solely based on it. In contrast, although RecMind-SI proposed the same alternative options in step 3, it takes into account the thought, action, and observation from both options 3 (1) and 3 (2) to generate the thought for the next step

Experiments: Effective and Generalizable



Traditional RS Baseline LLM RS Baseline RecMind Baseline RecMind + SI

Performance comparison in rating prediction on Amazon Reviews (Beauty) and Yelp.
Methods Beauty Yelp
RMSE MAE RMSE MAE
MF 1.1973 0.9461 1.2645 1.0426
MLP 1.3078 0.9597 1.2951 1.0340
AFM 1.1097 0.8815 1.2530 1.0019
P5 1.2982 0.8474 1.4685 1.0054
ChatGPT Rec 1.1589 0.7327 1.4725 1.0016
RecMind-CoT 1.1326 0.7167 1.3925 0.9794
RecMind-ToT (BFS) 1.1197 0.7059 1.3875 0.9766
RecMind-ToT (DFS) 1.1205 0.7103 1.3826 0.9774
RecMind-SI 1.0756 0.6892 1.3674 0.9698

BibTeX

Please kindly cite our paper if you use our code, data, models or results: 


@article{wang2023recmind,
  title={Recmind: Large language model powered agent for recommendation},
  author={Wang, Yancheng and Jiang, Ziyan and Chen, Zheng and Yang, Fan and Zhou, Yingxue and Cho, Eunah and Fan, Xing and Huang, Xiaojiang and Lu, Yanbin and Yang, Yingzhen},
  journal={arXiv preprint arXiv:2308.14296},
  year={2023}
}