Learning to explore (L2E): Deep reinforcement learning-based autonomous exploration for household robot

We study the autonomous exploration task in indoor environments for the mobile ground robot. We propose a three-stage exploration strategy: viewpoint generation, viewpoint scoring, and viewpoint selection, to make the algorithm agnostic to the robot’s planning and control modules. In particular, we propose the Learning to Explore (L2E) framework, which formulates the scoring and selection stages as a learning problem that could be solved by imitation learning (IL) and deep reinforcement learning (DRL). We use IL to pretrain the exploration policy and an off-policy DRL method to fine-tune it, improving the sample efficiency and accelerating the training process. We benchmark both the heuristic-based viewpoint scoring and selection methods and the proposed DRL method under the same exploration framework in realistic diverse indoor environments, and the results show that the L2E method can achieve 4% ∼ 22% minimum improvement when compared with baseline exploration approaches.