Large high-quality datasets of human body shape and kinematics lay the foundation for modelling and simulation approaches in computer vision, computer graphics, and biomechanics. Creating datasets that combine naturalistic recordings with high-accuracy data about ground truth body shape and pose is challenging because different motion recording systems are either optimized for one or the other.
MoVi, a Multipurpose Human Motion and Video Dataset, addresses this issue by using different hardware systems to record partially overlapping information and synchronized data that lends itself to transfer learning. This multimodal dataset contains 9 hours of optical motion capture data, 17 hours of video data from 4 different points of view recorded by stationery and hand-held cameras, and 6.6 hours of inertial measurement units data recorded from 60 female and 30 male actors performing a collection of 21 everyday actions and sports movements. The processed motion capture data is also available as realistic 3D human meshes. MoVi can be used for research on human pose estimation, action recognition, motion modelling, gait analysis, and body shape reconstruction.

Saeed Ghorbani, the project leader in MoVi, is a fourth-year Ph.D. student in Electrical Engineering and Computer Science at York University and also a VISTA trainee advised by VISTA Core researcher, Dr. Niko Troje. His research interests are machine learning, computer vision, computer graphics, and computer animation. His current research focuses on leveraging novel deep probabilistic models for realistic human motion modelling.

Check out Saeed's VISTA workshop video presentation for a brief introduction to the project:

To learn more about the project please visit: https://www.biomotionlab.ca/movi/

Image caption: Samples of an aligned video frame, joint locations, and estimated body in MoVi.