Towards Active Exploration and Flexible Goal Learning : Modeling Curiosity as a Distributed Cognitive process for Open-Ended learning in Robotics

Abstract

The topic of open-ended learning within the field of robotics presents a captivating and complex area of study. The purpose is to endow a robot with the ability to learn new skills autonomously by interacting with the environment. In practice, it generally involves modeling curiosity as a cognitive drive, so the robot explores the environment to discover and learn novel goals. However, current approaches to curiosity exhibit several limitations. Curiosity is modeled as a top-down mechanism that is responsible for the management of goals and tracking their learning progress. This centralized view contrasts with recent theories of cognition and limits the understanding of curiosity on a neural level. In general, this problem induces a lack of flexibility during the discovery and learning of goals.

To address these issues, this thesis proposes investigating the early formation and representation of goals by modeling sensorimotor contingencies (SMCs) with a physical robotic arm. The theory of SMCs is a bottom-up approach to cognition for the early formation of sensorimotor behavior. Through a series of experiments, the bottom-up approach demonstrated the possibility to learn SMCs with neural dynamics. In addition, an attentional mechanism, such as inhibition of return, guides the exploration of SMCs. Finally, a neural switch supports a flexible exploration/-exploitation trade-off for the learning of SMCs with dynamic neural fields.

Upon exploration of neural dynamics in SMCs, this research thesis then addresses the curiosity mechanism as a collection of decentralized and interconnected cognitive processes such as attention, habituation and cognitive persistence. Open-ended learning is formalized between the discovery and learning of goals based on neural dynamics. The goal discovery stage consists of detecting new stimuli and is regulated by a habituation paradigm, while goal learning is driven by prediction error and is modulated by a cognitive persistence mechanism. Then, a neural mechanism inspired by neuroscience research drives the robot toward goal discovery or learning. To test the architecture, several objects are placed in front of a simulated robot arm to determine its ability to autonomously discover and learn novel goals. The results demonstrated that the interactions among these cognitive processes improved exploration and increased learning flexibility toward objects.

Finally, this research examines the impact of neural plasticity on exploration by distinguishing random and direct exploration. By varying neural synaptic connections, a series of experiments with a real robotic arm demonstrated a progressive switch from random to direct exploration over time, thus showing a tendency toward uncertainty reduction. Furthermore, the results showed that neural plasticity influences exploration through the initial discovery of varied goals, which precedes a focus on detecting similar goals later on. This doctoral dissertation is an article-based compilation that includes five research studies, four of which have already been peer-reviewed and one article that is still pending review.