Report Number: CS-TR-95-1552
Institution: Stanford University, Department of Computer Science
Title: Embedded Teaching of Reinforcement Learners
Author: Brafman, Ronen I.
Author: Tennenholtz, Moshe
Date: June 1995
Abstract: Knowledge plays an important role in an agent's ability to perform well in its environment. Teaching can be used to improve an agent's performance by enhancing its knowledge. We propose a specific model of teaching, which we call embedded teaching. An embedded teacher is an agent situated with a less knowledgeable ``student'' in a common environment. The teacher's goal is to lead the student to adopt a particular desired behavior. The teacher's ability to teach is affected by the dynamics of the common environment and may be limited by a restricted repertoire of actions or uncertainty about the outcome of actions; we explicitly represent these limitations as part of our model. In this paper, we address a number of theoretical issues including the characterization of a challenging embedded teaching domain and the computation of optimal teaching policies. We then incorporate these ideas in a series of experiments designed to evaluate our ability to teach two types of reinforcement learners.