Agent Smith: Machine Teaching for Building Question Answering Agents

Building AI agents can be costly. Consider a question answering agent such as Jill Watson that auto-
matically answers students’ questions on the discussion forums of online classes based on their syllabi
and other course materials. Training a Jill on the syllabus of a new online class can take a hundred
hours or more. Machine teaching – interactive teaching of an AI agent using synthetic data sets – can
reduce the training time because it combines the advantages of knowledge-based AI, machine learning
using large data sets, and interactive human-in-loop training. We describe Agent Smith, an interactive
machine teaching agent that reduces the time taken to train a Jill for a new online class by an order of
magnitude.