Efficient search for robot skill learning: simulation and reality

Abstract

Table lookup with interpolation is used for many learning and adaptation tasks. Redundant mappings capture the important concept of "motor skill" in real, behaving systems. Few robot skill implementations have dealt with redundant mappings, in which the space to be searched to create the table has much higher dimensionality than the table. A practical method for inverting redundant mappings is important in physical systems with limited time for trials. The authors present the "Guided Table Fill In" algorithm, which uses data already stored in the table to guide search through the space of potential table entries. The algorithm is illustrated and tested on a skill learning task using a robot with a flexible link. The authors' experiments show that the ability to search high dimensional action spaces efficiently allows skill learners to find new behaviors that are qualitatively different from what they were presented or what the system designer may have expected. Thus the use of this technique can allow researchers to seek higher dimensional action spaces for their systems rather than constraining their search space at the risk of excluding the best actions. The authors also present a model for the robot arm, flexible link dynamics, and release mechanism of their robot. The authors' experiments suggest that the use of even a crude simulation model can be helpful for learning on the real robot.