The Deep Mind team then went a step further by creating a deep reinforcement learning agent to investigate whether or not the resulting artificial neural network was indeed capable of supporting navigation. As the Deep Mind team explained, “This agent performed at a super-human level…and exhibited the type of flexible navigation normally associated with animals, taking novel routes and shortcuts when they became available”. An example demonstrating these abilities is illustrated, below. In this example, the agent was trained in a maze with 5 doors when all but door #5 were closed (a). During testing, all doors were opened (b) and the agent successfully found shortcuts to the desired destination.
In my opintion, the most interesting features of this work included the use of two different neural network architectures in a single study:
- An RNN to develop a model of a portion of a mammalian brain that spontaneously mimicked the grid-cell structure of actual mammalian brains, and
- A deep reinforcement learning, agent-based system to explore how the resulting network enables velocity-vector-based navigation.