AskJill in SkillSync: Explanation and Trust in AI

SkillSync is an NSF-funded effort engaging an industry-academia consortium led by EduWorks and including Georgia Tech, the University System of Georgia, Credentials Engine, and several other partners. The goal is to build an AI application that enables companies to succinctly express reskilling needs for their workforce and helps colleges describe their educational offerings and respond to companies’ needs. The interactive tool better aligns job requirements and opportunities with reskilling knowledge, skills, and abilities, and distills these reskilling needs to universities in a way that matches specific learning outcomes.

When using SkillSync, both employers and educators can ask questions to a virtual coach named AskJill that is based on Georgia Tech’s virtual teaching assistant Jill Watson. AskJill helps explain SkillSync’s vocabulary, functions, and processing, thereby making it more transparent and trustworthy to its users. AskJill in SkillSync is a collaboration between the Design & Intelligence Laboratory and C21U.

GAIA: Adaptive Game-Playing Agents

The designs of long-living interactive games evolve through many versions. Changes from one version of a game to the next are typically incremental and often very small. A game designer (or a team of game designers and software engineers) formulates the requirements of the new version of the game, adapts the software for playing the previous versions to meet the new requirements, and implements and evaluates the modified designs of the game and the software. Typically the game designer uses high-level scripting languages to define the game environment (e.g. percepts, actions, rules, constraints) as well as the behaviors of various virtual agents in the game.

We posit that an interesting research issue in game playing is how a virtual agent might adapt its design, and thus its behaviors, to very small changes in its game environment. If the changes in the game environment can be arbitrarily large and complex then this becomes an “AI-complete problem.” However, even if the changes to the game environment are incremental and very small, this is a hard computational problem because changes to the environment can be of many types, modifications to the agent design can be of many types, there is no one-to-one mapping between changes to the environment and modifications to the agent design, and any modification to the agent design needs to be propagated down to the level of program code so that the new software is directly executable in the game environment.

The goal of this project is to develop an interactive environment called GAIA (for Game Agent Interactive Adaptation) in which the game designer generates requirements for a new version of a game, and the designer and the legacy software agents from previous versions of the game cooperatively adapt the agent designs to the new game requirements. We are developing and testing our meta-reasoning technique for adapting a mature program in the domain of turn-based, multi-player, strategy games (specifically FreeCiv).



Jill Watson: A Suite of Online Learning Tools

From Left to Right: Ida Camacho, Qiaosi Wang, Rohit A Mujumdar, Ashok Goel, Eric Gregori, Varsha Achar

With the advent of Georgia Tech’s OMSCS (a massive online course to earn a Masters in CS), thousands of people around the world have been enrolling to learn. An increase in learners implied more questions being asked on class discussion forums. With a limited number of instructors per course, answering questions became increasingly difficult. This is when Jill Watson was conceived.

Jill Watson is a Virtual Teaching Assistant, that can answer questions about a course syllabus, when deployed on online communication forums like Piazza or Slack. 

What initially began as a question answering agent for course syllabi, Jill Watson is gradually permeating into other domains. Jill Watson can now answer questions about VERA‘s instructional manual.

Watch the video below to learn more about Jill Watson!

VERA: Virtual Ecological Research Assistant

VERA-Ecol (Ecology)

The Virtual Ecological Research Assistant (VERA) is developed by the Design & Intelligence Lab at Georgia Tech, in collaboration with Smithsonian’s Encyclopedia of Life Department. The VERA system leverages AI technologies like Natural Language Processing (NLP) and AI compiler to support cognitive aspects of learning such as inquiry-based learning and model-based reasoning

VERA enables users to construct conceptual models of ecological systems and run interactive model simulations. This allows users to explore ecological systems and perform “what if” experiments to either explain an existing ecological system or attempt to predict the outcome of future changes to one.

Figure 1 shows a conceptual model of a food web that involves cougars (​Puma concolor) mule deer (Odocoileus hemionus​), domestic horses (e​quus caballus​), and cat grass (​dactylis glomerata​) (base population) generated from VERA.


Figure 1. Conceptual model of the relationships between species.

Figure 2 shows a simulation result generated from the conceptual model in Figure 1. In the simulation graph, you will see how the populations change over time in relationship to one-another. In particular, you will notice a stable predator-prey cycle between cougar (orange line) and mule deer (light blue line); that is, the cycles crest and fall in one after the other.



Figure 2. Simulation result when running the model from Fig. 1

Jill Watson, the AI-enabled virtual teaching assistant (TA), answers user questions about VERA. The covered questions include technical questions about the tool – “How do I add a new project” – as well as subject matter questions – “What is consumption rate?”

Please visit to sign up (with your GT account) and start exploring VERA.

VERA-Epi (Epidemiology)

The VERA team modeled the impact of social distancing on the spread of COVID-19. The simulations show that practice of social distancing slows the spread of COVID-19 (“flattens the curve”).

With VERA, you can explore parameter values and see for yourself the flattening of the curve. Interested in learning more? Visit VERA-Epi or read the white paper.

Intelligent Biologically Inspired Design

In Biologically Inspired Design, engineers use biology as a source of ideas for solving engineering problems. However, locating relevant literature is difficult due to vocabulary differences and lack of domain knowledge. IBID is an intelligent search mechanism that uses a functional taxonomy to direct search and a formal modeling notation for annotating relevant search targets.


For aspiring start-up entrepreneurs, developing their Business Model Canvas (BMC) well is key to their future success. Errol is an interactive agent that aims to play the role of a teacher who can provide feedback on users’ BMCs. Using some heuristics and applying these through the methods of lexical and semantic analysis, Errol poses questions to users based on their BMCs. Users can then think about the answers to these questions and update their BMCs accordingly. And on an even more general scale, we can say that Errol may be a way to develop theories on how humans make errors and what we can do to fix those.

VITA: Visual Thinking

Autism is a developmental disorder characterized by atypical behaviors in three different areas: social interaction, communication, and stereotyped or repetitive patterns of behavior and interests. Classic autism is one of several related conditions that comprise Autism Spectrum Disorders (ASDs), also known as Pervasive Developmental Disorders (PDDs). While the specific causes of autism are unknown, its etiology can be traced through genetic and environmental factors, brain development and functioning, cognition, and finally, behavioral manifestations.

We hypothesize that certain individuals on the autism spectrum exhibit a bias towards using visual instead of verbal mental representations for various tasks. This “thinking in pictures” form of cognition has been introspectively reported by many individuals on the autism spectrum, such as Temple Grandin, a high-functioning adult with autism who has written several autobiographical books touching on this notion.

Evaluating this hypothesis raises several research questions, for instance: 1) Do some individuals on the autism spectrum think in pictures, and if so, how can we identify them? 2) What kinds of visual mental representations are being used, how are they different, if at all, from neurotypical visual representations, and how are they recruited to perform various cognitive tasks? 3) What advances can be made in assessment, communication, and education, given this information about types of mental representations?

We are addressing these and other questions through a combination of designing behavioral studies intended to tease apart differences in mental representations and using computational models to make and evaluate precise specifications of various representational forms. Our long term goals include developing a set of design principles that can be used to construct tools for assessment and education for the autism community.

Biologically-Inspired Design


Design by Analogy to Nature Engine


Biologically inspired design, a kind of design by analogy, requires that engineers understand complex biological systems as analogues for design. A number of typical characteristics make biologically inspired design an especially interesting problem to study. (1) By definition, biologically inspired design is based on analogies requiring expertise across two disparate domains (e.g. architecture and biology), and thus is inherently interdisciplinary. (2) Since the objects, relations and processes across domains are very different, design collaborators speak from different lexicons. (3) Since biologists in general seek to understand the functions of designs occurring in nature whereas designers generally seek to generate designs for human needs, they use different methods of investigation and have different perspectives on design. (4) The resources, such as materials and processes, available in nature to realize an abstract design concept are very different from the resources available in the human domain.

Despite many pioneering and valiant efforts, the fact is that at present there is no science of biologically inspired design, and its practice remains scattered, empirical, and ad hoc. This raises many basic questions such as: How do biologists and engineers work together in teams? How do they generate new design ideas? How do they understand biological systems? What external knowledge representations of biological systems may help deepen their understanding? What should we teach in courses on biologically inspired design and how should we teach it? While these questions are critical to developing a science of biologically inspired design, they also provide a great opportunity to explore fundamental questions about cognition.




Image Credit:

The “Biologically-Inspired Design Project Banner” image on our website incorporates the following works. All of these listed works are licensed under Creative Commons CC BY 2.0:

MILA: Systems Thinking

The Systems Thinking project at the Design & Intelligence lab broadly focuses on teaching students at various levels to engage in thinking about complex, and especially ecological, systems. The project specifically addresses multiple specific tasks and abilities:

  • Helping students understand complex systems through the construction of structure-behavior-function models (Goel et al. 2010; Goel et al. 2013).
  • Providing feedback on model construction through dynamically-generated invokable simulations based on conceptual models (Vattam, Goel, & Rugaber 2010; Joyner, Goel, & Papin 2014).
  • Facilitating the broader process of scientific inquiry, grounded in model construction (Joyner, Majerich, & Goel 2013).
  • Providing personalized feedback on modeling and inquiry through metacognitive tutoring agents (Joyner & Goel 2014).

The Systems Thinking project has been under active development for over ten years. Over one thousand students have used the tools constructed within the project. The first of these tools, the Aquarium Construction Toolkit (ACT), allowed students to construct structure-behavior-function models of aquaria based on investigation of a classroom aquarium. The second tool, the Ecological Modeling Toolkit (EMT), built and expanded on ACT by generalizing the construction of models out into various ecological systems, such as ponds, lakes, and estuaries. The third and current tool, the Modeling & Inquiry Learning Application (MILA), augments the modeling process with multiple tools for inquiry to facilitate model-based inquiry in addition to modeling alone.