Video-Image-Sentence Multi-Modality Sequential Recommendation Model

At present, recommendation systems have become an indispensable
tool for users to access information. Traditional sequential recommendation systems often rely on explicit item IDs, which have limitations in data sparsity and
cold start scenarios. Recent studies have focused on using the modal features of
items as inputs to models, allowing knowledge learned from different modal datasets to be transferred. In this context, we propose a pre-training method for
modeling multiple modalities of data, which can effectively integrate information
from different modalities. We also introduce a new loss calculation method to
measure the performance of this method. Finally, to further improve the retrieval
performance of the model, we propose a new sequential recommendation method
that uses a sequence encoder to capture user interaction sequences and a project
encoder to encode project information, sharing parameters to enhance information. We evaluate the proposed methods on three public datasets and conduct
experiments, the results of which demonstrate an improvement in the performance of our methods.