Skeleton-Based Actions Recognition with Significant Displacements

In the realm of human skeleton-based action recognition, the graph convolu-tional networks have proven to be successful. However, directly storing co-ordinate features into the graph structure presents challenges in achieving shift, scale, and rotation invariance, which is crucial for actions with signifi-cant displacements. Such as figure skating, due to the significant displace-ments of athletes relative to the camera and the inherent perspective effects, leading to variations in scale, position, and rotation-related features. Signifi-cant displacements and perspective effects in actions video result in varia-tions in scale, position, and rotation-related features. To address this, drawing inspiration from leveraging high-order information, we propose a novel co-sine stream. This stream utilizes the bending angle of human joints for action recognition based on human skeleton. Furthermore, we introduce a new keyframe downsampling algorithm that significantly improves model per-formance. Notably, our approach does not necessitate any modifications to the backbone. Through extensive experiments on three datasets—FSD-10, FineGYM, and NTU RGB+D, our approach demonstrates improved recogni-tion of actions with significant displacement compared to current mainstream methods.