Key frame extraction based on sparse coding with deep frame features

Key frame extraction based on sparse coding can present the entire video with a small number of key frames, reducing the redundancy of the video. However, existing sparse coding-based methods use raw video frame features, which leads to high computational complexity and significant time consumption. In this paper, we propose a novel key frame extraction method based on sparse coding and deep frame features KSC-DFF to address these challenges. First, we obtain deep frame features using a deep neural network, which can reduce the dimensionality of the input video data and generate deep frame features such as the main object features of the frame. To automatically extract deep frame features, a YOLO-based deep neural network called YOLO-MLP was designed for video feature extraction. Then, we used sparse coding to extract key frames based on deep frame features, which can reduce information redundancy and computation time while maintaining high accuracy. Experimental results on SumMe demonstrate that the proposed KSC- DFF outperforms the existing methods with an increase of 49.4 and a time reduction of nearly 98 compared to the conventional sparse coding-based method SMRS.