A new version of Jill Watson, developed by Georgia Tech’s Design Intelligence Lab (DILab) and AI-ALOE, significantly enhances the online classroom experience. It outperforms OpenAI’s own assistant in educational settings by delivering more accurate and safer responses.
Key Improvements
- Jill answers student questions with high accuracy and boosts “teaching presence”, a crucial factor in effective online learning.
- The assistant now engages in context-rich, extended dialogue, pulling from courseware, textbooks, video transcripts, and more.
- System architecture includes:
- Preprocessed knowledge base
- MongoDB memory for conversation history
- Question classification, content retrieval, and moderated response pipeline
- Response validation using textual entailment
In controlled experiments, Jill Watson achieved 75%–97% accuracy vs. OpenAI’s Assistant at ~30%, students with Jill perceived better teaching and social presence, and showed improvement in grades (e.g., more A’s, fewer C’s).
- Safety & accuracy:
- Jill: 78.7% accurate, only 2.7% harmful errors
- OpenAI Assistant: 30.7% accurate, 14.4% harmful errors
In short, Jill Watson is setting a new standard for AI in online education. More accurate, safer, and more effective at supporting both teaching and learning than generic AI chatbots.
