Deep Multimodal Purchase Decision Modeling for E-Commerce Recommendation

Abstract

Understanding customers' purchasing decisions is a fundamental challenge in e-commerce. This work presents a multi-modal deep learning architecture using product image, product description, and user behavior history to predict the probability that a user will purchase a product. We use ResNet-50, BERT, and a bidirectional LSTM to encode features from the three modalities and propose a cross-modal attention mechanism to integrate the features. Our experiments are carried out on the Amazon Electronics dataset. We achieve aROC-AUC of 0.892, which outperforms the best unimodal model by at least 8%. Ablation experiments reveal that the different modalities complement one another, with user behavior history being the most important modality.The results show that combining visual, textual, and behavioral information can provide a more realistic understanding of customer preferences and improve the quality of recommendation decisions. This study also highlights the importance of designing recommender systems that are not only accurate, but also capable of capturing the complexity of real online shopping behavior.