Learning by Imitation and Exploration: Bayesian Models and Applications in Humanoid Robotics

Many existing methods for planning robotic actions require an engineer to explicitly model the complex physics of the robot and its environment. This process can be costly, tedious, error-prone, brittle, and inflexible to changes in the environment or the robot. The method I propose involves learning a predictive model of the robot's dynamics, represented directly in terms of sensor measurements, solely from exploration. Experiments are performed with a Fujitsu HOAP-2 25-degrees-of-freedom humanoid robot and the Webots dynamic simulation software. I present results demonstrating that the robot can learn dynamically stable, full-body imitative motions simply by observing a human demonstrator and performing explorative learning. Additional results show how the inference-based action selection technique can be used for policy learning, where sensory feedback can be used to adapt behavior online. I present policy learning results for a lifting behavior (learned via imitation) that generalizes to a wide range of objects of novel, unknown density. Besides imitation-based learning, this dissertation makes other contributions to the emerging area of robotic learning. First, intractability due to very high-dimensional state and control spaces is tackled using dimensionality reduction techniques. Second, nonparametric techniques are introduced to handle the problem of learning and inference with continuous-valued random variables.