Real-Time Human Motion Capture and Animation in Blender 3D Models using AI-based Pose Estimation

Abstract

Traditional human motion capture systems often require expensive hardware and complex setups, limiting accessibility and lacking the ability to animate human models in real-time without additional equipment. Animating characters in real-time with accurate motion demands seamless integration between pose recognition and animation systems. This paper presents a real-time human motion capture and animation system using AI-based pose estimation. Leveraging Media Pipe Pose Landmark detection, the proposed system identifies 33 body landmarks from a mobile app built in Java and Kotlin. The app calculates various poses using the detected landmarks and sends them to a Firebase Realtime Database. A Python addon in Blender retrieves the pose data in real-time, adjusting the 3D model’s bones and rotations based on the detected poses, enabling live character animation. The integration of Firebase for real-time synchronization and Blender’s armature system offers a cost effective and flexible approach to motion capture. This solution eliminates the need for expensive hardware, providing a portable method for high-quality animation creation. Significant potential for applications in gaming, virtual reality, and animation studios. Experimental evaluation demonstrated an average pose estimation accuracy of 80% and a real-time latency of approximately 120 ms, confirming the system’s efficiency for smooth motion transfer. While the current implementation is optimized for single-user motion tracking, future work will focus on multi person tracking and integration with advanced 3D environments for improved realism.