Long-horizon manipulation through hierarchical motion planning with subgoal prediction

The research on long-horizon manipulation in environments with numerous objects and subtasks falls under the framework of task and motion planning (TAMP). One effective solution for TAMP is to separate higher-level discrete short-horizon subgoals, and lower-level continuous motion generation to enhance robustness, scalability, and generalizability. We propose a concept of hierarchical framework combining deep neural networks (DNN) for higher-level subgoal decisions and optimization for lower-level motion control. This will be evaluated on a latent state box transport and stacking task –where the robot needs to change the order of actions and speed to control during motion execution. Additionally, we can apply this framework to daily tasks such as cooking, where the robot needs to recognise the states of ingredients, select appropriate tools and subtasks, and adjust its motions accordingly.