Automatic State Abstraction from Demonstration
Luis C. Cobo, Peng Zang, Charles L. Isbell and Andrea L. Thomaz
Learning from Demonstration (LfD) is a popular technique for building decision-making agents from human help. Traditional LfD methods use demonstrations as training examples for supervised learning, but in complex tasks this can require more examples than it is practical to obtain. We present Abstraction from Demonstration (AfD), a novel form of LfD that uses demonstrations to infer state abstractions and reinforcement learning (RL) methods in those abstract state spaces to build an agent. This combination of demonstrations and RL can solve decision problems more effectively than either alone. Empirical results show that AfD is greater than an order of magnitude more sample efficient than just using demonstrations as training examples, and exponentially faster than RL alone.