Motion PointNet: Solving Dynamic Capture in Point Cloud Video Human Action

Abstract

Motion representation plays a pivotal role in understanding video data, thereby elevating the dynamic capture to the forefront of action recognition tasks based on point cloud video. Previous works mainly compute the motion information in an unguided way, e.g. aggregate the spatial variations on adjacent point cloud frames using 4D convolutions or capture a point trajectory with kinematic computation like scene flow. However, the former fails to explicitly consider motion representation in corresponding frames, and the latter's reliance on tracking point trajectories becomes impractical in real-life applications due to the potential inter-frame migration of points. In this paper, we tackle the dynamic capture in point cloud video action by formulating it as solvable partial differential equations (PDEs) in feature space. Based on this intuitive design, we propose Motion PointNet, a novel method that improves the dynamic capture in point cloud video human action by constructing clear guidance for network learning. Motion PointNet is composed of a lightweight yet effective PointNet-like encoder and a PDEs-solving module for dynamic capture. Remarkably, our Motion PointNet, with merely 0.72 M parameters and 0.82 G FLOPs, achieves an impressive accuracy of 97.52 % on the MSRAction-3D dataset, surpassing the current state-of-the-art in all aspects. The code and the trained models will be released for reproduction.