Learning an accurate model of the environment is essential for model-based control tasks. Existing methods in robotic visuomotor control usually learn from data with heavily labelled actions, object entities or locations, which can be demanding in many cases. To cope with this limitation, we propose a method, dubbed DMotion, that trains a forward model from video data only, via disentangling the motion of controllable agent to model the transition dynamics. An object extractor and an interaction learner are trained in an end-to-end manner without supervision. The agent's motions are explicitly represented using spatial transformation matrices containing physical meanings. In the experiments, DMotion achieves superior performance on learning an accurate forward model in a Grid World environment, as well as a more realistic robot control environment in simulation. With the accurate learned forward models, we further demonstrate their usage in model predictive control as an effective approach for robotic manipulations.

Paper: https://arxiv.org/abs/2103.04301
Project: https://hyperplane-lab.github.io/dmotion/

The 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021) will be held in Prague, Czech Republic online September 27 – October 1, 2021. IROS 2021 aims to bring a truly historical event to represent the first-ever conference organized by a Central European country and, more remarkably, by the country that introduced the word "robot” to the world. The forthcoming IROS 2021 conference targets creating opportunities to meet, discuss, and become familiar with the most recent advances in the field of robotics, autonomous and intelligent systems, human-compliant robots, medical robotics, and many more. IROS 2021, as ever before, will include an on-line robotic exhibition as an essential part of the event.