Imitation Learning in Relational Words: A Functional Gradient Boosting Approach
Sriraam Natarajan, Saket Joshi, Prasad Tadepelli, Kristian Kersting and Jude Shavlik
Imitation learning refers to the problem of learning how to behave by observing a teacher in action. Prior work has addressed this problem in relational worlds, in which there is a varying number of objects and relations among them. Essentially, simple relational policies are learned by viewing imitation learning as supervised learning of a function from states to actions. For propositional worlds, however, functional gradient methods have been proven to be beneficial. They are simpler to implement than most existing methods, more efficient, more naturally satisfy common constraints on the cost function, and better represent our prior beliefs about the function's form. Building on recent generalizations of functional gradient boosting to relational representations, we implement a functional gradient boosting approach to imitation learning in relational worlds.In particular, given a set of traces from the human teacher, our system learns a policy in the form of a set of relational regression trees that additively approximate the functional gradients. The use of multiple additive trees combined with relational representation allows for learning more expressive policies than what has been done before. We demonstrate that the learner is capable of imitating the user effectively in several different domains such as blocksworld, traffic signal domain and resource gathering.