A new version of Jill Watson, developed by Georgia Tech’s Design Intelligence Lab (DILab) and AI-ALOE, significantly enhances the online classroom experience. It outperforms OpenAI’s own assistant in educational settings by delivering more accurate and safer responses.

Key Improvements

• Jill answers student questions with high accuracy and boosts “teaching presence”, a crucial factor in effective online learning.
• The assistant now engages in context-rich, extended dialogue, pulling from courseware, textbooks, video transcripts, and more.
• System architecture includes:
  • Preprocessed knowledge base
  • MongoDB memory for conversation history
  • Question classification, content retrieval, and moderated response pipeline
  • Response validation using textual entailment

In controlled experiments, Jill Watson achieved 75%–97% accuracy vs. OpenAI’s Assistant at ~30%, students with Jill perceived better teaching and social presence, and showed improvement in grades (e.g., more A’s, fewer C’s).

• Safety & accuracy:
  • Jill: 78.7% accurate, only 2.7% harmful errors
  • OpenAI Assistant: 30.7% accurate, 14.4% harmful errors

In short, Jill Watson is setting a new standard for AI in online education. More accurate, safer, and more effective at supporting both teaching and learning than generic AI chatbots.

DILab’s Karen Taneja presented the paper, “Towards a Multimodal Document-Grounded Conversational AI System for Education”, at the 26th International Conference on Artificial Intelligence in Education in Palermo, Italy.

Artificial intelligence is only beginning to emerge in higher ed. Its unique ability to process large data sets, speed up responses, make predictions, and learn has the power to transform higher ed. As AI becomes more integrated in colleges and universities, what impact will this technology have on the way colleges teach, conduct research, and manage their operations?

As part of a two-part series of virtual forums examining the role of AI in higher ed, a panel of leading thinkers and practitioners in the use of AI will join The Chronicle for a discussion of AI and machine learning in higher ed’s future, including such issues as:

• How will AI benefit instruction, for both in-person and online classrooms?
• How can it help bridge equity gaps in learning?
• How can AI-powered virtual tutors help level the playing field?

With Support From University of Florida

Panelists will include:

• Fred H. Cate, Vice President for Research, Indiana University
• Dan Garcia, Teaching Professor in the Electrical Engineering and Computer Science Department, University of California at Berkeley
• Ashok Goel, Professor of Interactive Computing and Chief Scientist of C21U, Georgia Institute of Technology
• Marsha C. Lovett, Associate Vice Provost for Teaching Innovation and Learning Analytics, Carnegie Mellon University
• Morris Thomas, Director, Center for Excellence in Teaching, Learning, and Assessment, Howard University

The Silver Lining for Learning podcast featured the NSF Institute for Adult Learning and Online Education and DILab’s technology, Jill Watson, on their newest episode.

The National AI Institute for Adult Learning and Online Education (AI-ALOE) is a research institute led by the Georgia Institute of Technology.  AI-ALOE is a leader in developing cutting-edge AI applications for improving online adult education. These efforts are bidirectional – AI is used to transform online adult learning and online adult learning to transform AI. These transformations are not “just doing things better” but “doing better things” in effectiveness, efficiency, access, scale, and personalization.

The podcast focuses on a generative-AI-based virtual teaching assistant, Jill Watson. Developed by researchers at AI-ALOE, Jill Watson is deployed on the class learning management system to engage in extended, cognitively-rich and factually grounded one-on-one conversations with students about topics in instructor-approved course materials anytime and anywhere. With its reach, Jill Watson affords opportunities for improving and democratizing post-secondary learning.

Special guests Ashok Goel, Sandeep Kakar, and Kristy Singletary

Congratulations to all of DILab’s graduates this semester! We always want to acknowledge the hard work done by team members and celebrate their successes!

DILab graduates this semester include: Chelsea Wang, Dinesh Ayyappan, and Jolanta Wojcik

Chelsea Wang formally received her PhD degree for Mutual Theory of Mind

Dinesh Ayyappan (Personalization of Learning in VERA) graduated with an OMSCS degree

Anthony Chen (Scalability of Jill Watson) graduated with an OMSCS degree

Jolanta Wojcik (Impact of SAMI on Belongingness) graduated with an OMSCS degree

November 12, 2024 — Chronicle Intelligence, a division of The Chronicle of Higher Education, recently published a comprehensive report on AI in Education, featuring Ashok Goel and AI-ALOE – AI Institute for Adult Learning and Online Education.  Colleges are grappling with the role of artificial intelligence (AI) in education as AI tools become more prevalent on campuses. Administrators see AI as a powerful tool to improve efficiency and enhance teaching, while skeptics raise concerns about its potential to dehumanize education and affect students’ development in writing and research.

October 1, 2024 – The article explores the transformative impact of Jill Watson, an AI teaching assistant introduced at Georgia Tech, designed to support students in a large online course by answering routine queries.

Student feedback has been overwhelmingly positive, with many considering nominating Jill for an Outstanding TA award due to her quick and relatable responses. Moreover, her implementation demonstrated how AI can complement human educators by taking on repetitive tasks, allowing them to focus on deeper educational interactions. This collaboration highlights the potential of AI in education, suggesting that it empowers rather than replaces human instructors.

October 12, 2024 – Listverse published 10 Times AI Replaced Humans (and No One Noticed) and Jill Watson is number 5!

The article recaps DILab’s Professor Ashok Goel introducing Jill Watson in his online artificial intelligence course. Jill efficiently responded to student questions, graded assignments, and participated in discussions. It wasn’t revealed until the end of the semester that Jill was an AI created with IBM Watson technology. Her successful integration into the classroom highlighted the potential for AI to assist in education and raised important questions about the role of AI in enhancing personalized learning experiences.

September 24, 2024 – Jill Watson and DILab director, Ashok Goel, were featured in the Fast Company Article “These schools don’t fear artificial intelligence. They put it to work”. The article discusses advancements in AI tutoring systems, particularly at Arizona State University, Harvard University, and Georgia Institute of Technology.

In addition, the article highlights a project called SAMI, which uses AI to foster community by pairing students based on shared interests, thereby improving their sense of belonging—important for emotional and cognitive well-being.

September, 13, 2024, DILab’s Qiaosi (Chelsea) Wang passed her PhD Defense of Dissertation! Chelsea presented her thesis on Mutual Theory of Mind for Human-AI communication which she presented at the CHI conference earlier this year. We are incredibly proud of all the effort and dedication Dr. Wang has put into her thesis and are looking forward to the extraordinary work she will accomplish in the future.