March 18, 2020
How to Work at Home and Get Stuff Done
School of Interactive Computing, Georgia Institute of Technology, email@example.com
A Little About Myself for Reference
I am 49 years old with a wife, 2 teenage boys (1 driving), and 2 dogs. I have worked as either a software engineer or an engineering manager for 25 years of which 17 of those years have been work at home. I earned a master’s of computer science from the Georgia Institute of Technology’s OMSCS online learning program. I currently work remotely as a research scientist for Georgia Tech.
Working From Home is Very Different From Working in an Office
Whether you sit in front of a computer at home or you sit in front of a computer in an office the actual work you are doing is the same. At the task level, the job does not change. This is a very important concept to remember; you are paid to complete tasks, not work hours. When working at home the environment changes, communications change, but the tasks (and the need to complete tasks) stay the same. Unfortunately, your home is comfortable, full of distraction, and a barrier to communications with your manager and coworkers. All of which work against you when you are trying to get work tasks (stuff) done.
Your Home has Been Designed for Comfort, not Productivity
The first obstacles to getting stuff done at home are the couch, TV, and refrigerator. These distractions were purchased for comfort, to wind down after a day or week of hard work. These items will call to you when you are trying to get stuff done. The solution is a workspace out of sight of those distractions; preferably, a dedicated workspace with a door . When it is time to work, go to your workspace and if you have a door close it. Your home workspace should be designed for productivity by following the same rules as an office workspace in terms of layout and amenities.
I have a wife and two teenage boys that I love very much, but they keep me from getting stuff done by assuming that if I am home I am available. If you have a door in your workspace, close it, and when you are on a conference call put a sign on the door stating that you are on a video call and are not to be disturbed. Invest in a pair of comfortable noise-canceling headphones with a microphone. If you do not have a door, explain to your family that at any time you may be on a conference call and to approach you assuming you are on a call. I also use hand signals to signal to my family that I am on a call.
Out of Sight, Out of Mind – Communication is key
In my experience working at home as a manager and an individual contributor, this proverb is just applicable now as it was in 850 B.C. How do you get stuff done if you are not sure what the stuff is or you are unsure of the definition of “done”. Most people do not realize the amount of important information that is conveyed via impromptu hallway meetings until they have worked at home for a month or two. “Out of sight, out of mind” is human nature  and must be actively counteracted in order to understand what stuff needs to be done.
Over-communicating with your manager, your teammates, and your internal customers is the solution to, “out of sight, out of mind”. Check your email often during your manager’s business hours, be available on Slack, strive to respond to communications immediately. This is very easy to do by installing email and Slack on your phone. I consider it an important part of my job to keep my manager and team informed about what is going on, so I tend to send a lot of emails. This is very dependent on your manager; some of my past manager’s liked the fact that I kept them in the loop via email and others would rather be updated during conference calls. Make sure you figure out how your manager likes to communicate. Even if you have a manager that would rather be updated via a call, send an agenda email before the call with your update. In my experience, you are always better off over-communicating than under communicating when working at home.
Although over-communicating can do a lot to combat “out of sight, out of mind”, research has shown there is a potential work from home promotion (WFH) ‘discrimination’ penalty . This is situation-specific in my experience as both a manager and an individual contributor. Working at home needs to be factored into your career goals with the understanding that you may not have the same career advancement options as someone who works in the office.
The Advantages of Working at Home
If done right, research has shown that working from home (WFH) is beneficial to both the employee and the employer . Success requires a unique manager and employee who are both invested in making WFH work. The manager must be willing to trust and give up a little control, while the individual contributor must be willing to spend more time communicating.
The reward for the company is a more productive employee; research shows WFH employees are more productive than office employees . The reward for the WFH employee is flexible hours, no commute, and a flexible work environment. For example, I enjoy working at night so I split my day between meetings during business hours and coding at night; sleep flexibility is another advantage of working from home.
 Crosbie, Tracey, and Jeanne Moore. “Work–life balance and working from home.” Social Policy and Society 3.3 (2004): 223-233.
 Bloom, Nicholas, et al. “Does working from homework? Evidence from a Chinese experiment.” The Quarterly Journal of Economics 130.1 (2015): 165-218.
 Schindler, Andreas, and Andreas Bartels. “Parietal cortex codes for egocentric space beyond the field of view.” Current Biology 23.2 (2013): 177-182.
Author Biography: Eric Gregori is a Research Scientist in the Design Intelligence Lab at Georgia Institute of Technology. His research interest revolves around the use of Knowledge Engineering, Machine Learning, and Natural Language Understanding in education and customer support. He is currently working on the Jill Watson conversation agent to support education by amplifying the utility of teachers. He holds an MS in computer science from Georgia Tech.
March 17, 2020
Online Pedagogy: Lessons learned from teaching an online course in Georgia Tech’s OMCSCS program
Ashok K. Goel (firstname.lastname@example.org) Design & Intelligence Laboratory, School of Interactive Computing, Georgia Institute of Technology
The spread of COVID-19 has led to an unprecedented move towards online education across the country and around the world. It is too early to predict the long-term impact of this extraordinary move: it may be temporary and much of education may soon revert back to the old normal of in-person classes, or it may lead to a large-scale shift towards online learning. Even if the move is only temporary, it may lead to a new normal of blended learning in which online educational materials, home works and assessments support in-person learning. In any case, this raises fundamental questions for online pedagogy: How do we develop a successful online course? How do we prepare high quality educational materials for online classes?
In 2014, my Georgia Tech colleague David Joyner and I faced exactly these questions when we developed an online course on Knowledge-Based Artificial Intelligence (KBAI) as part of Georgia Tech’s Online Masters of Science in Computer Science program (OMSCS; http://www.omscs.gatech.edu/). We identified about 150 KBAI concepts, methods and skills we wanted students to learn, developed a set of 26 video lessons, and designed a suite of learning assessments including home works, design and programming projects, and take home examinations (Goel & Joyner 2016, 2017). In Fall 2014, we offered the online class to about 200 students in the OMSCS program. In Fall 2015, I transformed the pedagogy I had been using in the in-person KBAI class for graduate and undergraduate residential students for more than a decade into blended learning (Goel 2019). In Spring 2016, my Design & Intelligence research laboratory developed a virtual teaching assistant called Jill Watson for automatically answering routine questions on the discussion forum of the online KBAI class (Goel & Polepeddi 2017). We estimate that more than 6000 students have taken the online and blended versions of the KBAI class since Fall 2014 and more than 150 (human) teaching assistants have helped with the teaching and learning in the class.
Assessing the quality of learning in any class is a complex matter, whether the class is online, blended or in-person. Three types of data indicate that the quality of learning in the online KBAI class is comparable to that in the in-person class for residential students (Goel & Joyner 2016, 2017; Goel 2019). First, surveys of online students from Fall 2014 through Fall 2019 report the same kind and degree of satisfaction with the online KBAI class as do the residential students with the in-person KBAI class during the same period. Second, the completion ratio in the online KBAI classes during this period has been comparable to that in the in-person KBAI classes, which is atypical for online classes. Third, the performance of the online KBAI students on the same set of learning assessments has been comparable to that of the residential students in the in-person KBAI classes. To control for the difference in the student demographics between the online and residential students, in Fall 2018 and Fall 2019, we repeated the above quasi-experimental studies in online and in-person sections of the KBAI class offered only to residential students, and found similar results.
Design Principles for Online Classes
In Goel & Joyner (2016), we analyze the design principles for developing online classes in detail. Table 1 summarizes the 10 design principles underlying the online and blended KBAI classes. For example, the first principle suggests that the course instructor may want to explicitly the learning goals, outcomes, strategies and assessments before developing the online class. The eighth principle indicates that design of an online class is an iterative process, with each iteration based on feedback and reflection on the preceding iteration. Thus, it is important to collect and analyze feedback and leave time for deliberation and reflection.
Table 1: Design Principles for Online Classes (adapted from Goel & Joyner 2016)
Design Principles for Video Lessons
The 26 video lessons for the online and blended versions of the KBAI class embed about 150 exercises, one for each concept in the concept inventory, as well as about 100 tutors that provided adaptive feedback on many of the exercises (https://www.udacity.com/course/knowledge-based-ai-cognitive-systems–ud409). My colleague Chaohua Ou with Georgia Tech’s Center for Teaching and Learning has analyzed the design of the videos, and student responses to them over several semesters, in detail (Ou, Joyner & Goel 2019). Table 2 summarizes the seven principles for designing video lessons for online classes derived from the analysis. For example, the seventh principle suggests the use of prepared visuals rather than drawing them at run time.
Table 2: Design Principles for Video Lessons (adapted from Ou, Joyner & Goel 2019)
It is important to note three qualifications here. First, developing a successful online class requires expertise not only in the subject of the course, but also in information technology and learning science. It also requires a strong team, and significant financial and technological support. We were very fortunate to have all these assets in developing the online KBAI class in 2014. My research laboratory conducts research into KBAI and thus we had the expertise in the subject; we are computer scientists and thus we had some expertise in information technology as well; and Joyner and I already were developing expertise in learning science as part of his Ph.D. work. In addition, Georgia Tech generously provided financial and technological support for developing the video lessons for the online KBAI class.
Second, if a teacher’s goal is simply to convert the educational materials prepared for an in-person class to an online forum due to the spread of COVID-19 and the consequent need for social distancing, then we expect the design principles in Table 1 should be of immediate value. The principles in both Tables 1 and 2 are likely to add more value if in the longer term a teacher wants to develop a new online course from the start.
Third, learning in general is situated in the external world and thus is context dependent. The effectiveness and efficiency of learning depends not only on the individual learner, teacher, and educational materials, but also on the physical, technological, social and cultural contexts of learning. While we found the design principles enumerated above useful in developing the online KBAI class, we expect that their operationalization will vary across different learning contexts.
A. Goel. Preliminary Evidence for the Benefits of Online and Blended Learning. In A. Madden, L. Margulieux, R, Kadel & A, Goel (editors), Blended Learning in Practice: A Guide for Practitioners and Researchers, MIT Press. April 2019.
A. Goel & D. Joyner. An Experiment in Teaching Artificial Intelligence Online. International Journal for Scholarship of Technology-Enhanced Learning, 1(1): 1-27, 2016.
A. Goel & D. Joyner. Using AI to Teach AI: Lessons from an Online AI Class. AI Magazine, 38(2): 48-59, Summer 2017.
A. Goel & L. Polepeddi. Jill Watson, A Virtual Teaching Assistant for Online Education. In C. Dede, J. Richards & B. Saxberg (editors), Education at Scale: Engineering Online Teaching and Learning, NY: Routledge, 2018.
C. Ou, D. Joyner, & A. Goel. Designing and Developing Video Lessons for Online Learning: A Seven-Principle Model. Online Learning Journal, 23(2): 82-104, June 2019.
Author Biography: Ashok Goel is a Professor of Human-Centered computing in the School of Interactive Computing at Georgia Institute of Technology. He is also the Chief Scientist with Georgia Tech’s Center for 21st Century Universities. In 2014, he co-developed an online course on Knowledge-Based AI; in 2016, his research laboratory developed Jill Watson, a virtual teaching assistant for answering questions in online class discussion forums; and in 2019, he co-edited a volume on Blended Learning published by MIT Press. In 2017, Ashok received Georgia Tech’s Class of 1934 Outstanding Innovative Use of Educational Technology Award; in 2019, he received AAAI’s Outstanding AI Educator Award; and in 2020, he received the University System of Georgia’s Faculty Hall of Fame Award for Scholarship of Teaching and Learning.
In the article below, Ashok Goel discusses the question: will AI take over the world? He argues that intelligence, ethics and values, emotions and feelings, society and culture, all go together: there is little prospect for human-level or superhuman intelligence in a society or a species without correspondingly high-level ethics, emotions, and culture.
Last Monday (August 18, 2014) we released our online course on Knowledge-Based AI through Udacity as part of the Georgia Tech OMS in CS program. David Joyner, a learning scientist, is the course developer and TA for the course, and I am the course creator and instructor. The course is presently being taken by some 200 professionals.
The course is organized around a sequence of ambitious design and programming projects. It extensively uses problem-based learning, learning by example, learning by doing, and affords considerable opportunities for collaboration, reflection, and personalization. We think
of it as the first iteration in our program for design-based research on online learning.
Creating this course has been a lot of work. I must have put in >200 hours into it. David probably has put in >800 hours. Yet, I am excited about it. In the first five days, we received >800 postings on the collaboration forum for the course, indicating deep engagement. Many of the discussions are of high quality; Some are also quite intense. I expect to learn a lot from the course.
Quick: what’s the hardest thing you’ve ever learned to do?
I’ll speculate that for many of you, the answer was something that you learned in college or in your career: perhaps fully solving advanced differential equations or articulating the complex nuances of deconstructionist literary analysis. For others, it may be something earlier that posed a severe challenge to you at an earlier level of education; many students struggle with logarithms, comma splices, or Newton’s laws of motions. From my experience as a tutor, I can say with confidence that many difficulties I see with advanced math can be traced back to lingering difficulties students encountered with fractions back in middle school.
All those topics are very challenging, but I would make the case that another pair of skills is far, far more difficult: walking and talking. When we think about the series of muscle movements that one must complete to walk on two feet, the physics are astounding. It’s not surprising that bipedalism is so uncommon (though not strictly human) in the animal kingdom. Similarly, the process of combining syllables into words, words into sentences, and sentences into messages is almost indescribably complex. We often experience this complexity first-hand when attempting to learn second languages as adults. Yet, we all naturally learn both these incredibly complex skills within the first handful of years on earth.
In my eyes, the fact that we can learn such difficult abilities from such a young age points to the existence of a natural desire to learn, a suggestion echoed by many philosophers, scientists, and educators over the years. Carl Sagan writes that “Every child starts out as a natural-born scientist.” Empirical evidence suggests the same (Gopnik 2012). The evolutionary case for an innate desire to learn nearly writes itself as one can imagine that the proclivity to gain new skills may directly increase one’s chance of survival in changing circumstances and environments.
This posited love of learning hits a harsh reality, however, when we listen to the average teenager’s commentary about school. Traditional schooling has developed a bad reputation among students as boring and irrelevant. The word ‘homework’ elicits a groan, and the notion of ‘tests’ and ‘exams’ can still cause nightmares in those who have long-since completed that stage of life. But we established previously that children are born with a love of learning; how can this coexist with the negative attitude many students harbor toward school, whose stated purpose is learning?
Carl Sagan explains this with the second half of the previous quote, ” Every child starts out as a natural-born scientist… and then we beat it out of them.” Some students “learn” through their early experience that they are “bad” students and adopt a self-image wherein they cannot succeed in traditional education. Others, lacking challenge, “learn” that education is itself “boring” given how little stimulation they are allowed to receive before being slowed down to the pace of the class. Many students succumb to the temptations of extrinsic motivation and begin pursuing learning for the sake of the grade or the achievement, thus adopting a performance orientation focused more on positive judgments and external accolades than internal success and mastery (Dweck & Leggett 1988). When we couple this with the finding that introducing extrinsic motivators reduces intrinsic motivation (e.g. Deci 1971; Kruglanski, Friedman, & Zeevi 1971; Lepper, Green, & Nisbett 1973), this development undermines exactly the type of innate love of learning that ought to instead be fostered. Still for other students, an idea emerges in their mind that the pinnacle of success is to achieve without effort, and thus they adopt a fixed mindset that encourages avoiding situations where their weaknesses might be revealed, even though these are precisely the situations where learning may occur (Dweck 2006). Perhaps for these reasons or perhaps for others, it seems that sometime between the acquisition of language and the onset of midterms, that natural love of learning disappears.
But does it really disappear? No! This love of learning never truly goes away. For some, it retreats to the background, awaiting a setting when it can emerge safely without the fear of judgment. For others, it finds other outlets, as video games or other hobbies supply the personal context in which learning can be pursued at one’s own pace and for one’s own purposes. Although we may be conditioned to dissociate the joy of learning from the contexts in which learning is supposed to occur, the love of learning itself does not dissipate. Learning is too ingrained within our species to simply disappear due to a handful of negative experiences. Instead, it lies dormant like a volcano, calmly awaiting an opportunity to erupt back to the surface and reintroduce the learner to its flow.
This, to me, is the ultimate challenge and the ultimate opportunity for massively open online courses. These courses represent the opportunity to reengage that innate desire to learn. By their open nature, MOOCs facilitate self-driven learning; material can be consumed at the individual’s pace, challenging students without overwhelming them and supporting students without boring them. The absence of tuition and human graders also diminishes the extrinsic motivation to participate in a MOOC; typically, rather than a grade, you instead leave the course simply with the skills you’ve mastered and the work you’ve produced. To participate and succeed in a MOOC, you simply must be self-driven; there is no financial loss to failure, there is no schedule forcing continued participation, and there is no easily-quantifiable résumé bullet point. The drive to learn must come from within, as it did decades ago when you and I first learned to walk and talk.
Creating an environment in which those with the desire for lifelong learning succeed, however, is just the beginning. Such environments have always existed. The internet has created boundless opportunities for knowledge, and those with the highest levels of personal drive can already set out to teach themselves anything they might want to know. The challenge of MOOCs that complements the opportunity is exactly this: how do we reengage that innate desire to learn? Given the opportunity to help lifelong learners keep learning, how do we motivate students to come back to lifelong learning in the first place?
This is an enormous challenge, but it is not insurmountable. We must find ways to preserve, or reinstall, students’ intrinsic motivation to learn while maintaining engagement at that flow-inducing point where challenge meets ability (Csikszentmihalyi 1991). We can begin to address this latter half in part through the self-pacing facilitated by open courses, but also by borrowing the lessons learned by the intelligent tutoring system community on providing automated, situated, individualized feedback. (e.g. Anderson, Corbett, Koedinger, & Pelletier 1995). As for the former element, the project-based learning community helps us not only address the need for intrinsic motivation, but also the need for demonstrable skills. The personal approach afforded by project-based learning allows students to connect their unique interests to the learning goals of the course. Through project-based learning, students’ intrinsic motivation can be preserved as mastery of the skill is externalized not into a grade or a certification, but into an ability to be performed in a meaningful, personal context. As an added bonus as well, the project that results from this approach becomes a portfolio piece, a demonstrable, external indicator of the abilities that the student has mastered.
But what does this mean for you? The need for lifelong learning is more evident now than ever as foundational skills, especially in the technology industries, reinvent themselves seemingly every year. In today’s careers, continuing to succeed is dependent on continuing to learn. Many times, these learning goals will not necessarily correspond to specific projects or problems, but rather to the need to stay current in the industry as a whole, and as such, the learning must motivate itself. The goal of learning must be to learn rather than to achieve. Toward this end, seek out those learning experiences that most closely resonate with what you what to learn right now. Let your intrinsic motivation guide you to the first step, and as you relearn or reawaken the joy of learning, you may find that each learning experience after becomes easier and more fulfilling to embrace.
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The journal of the learning sciences, 4(2), 167-207.
Csikszentmihalyi, M. (1991). Flow: The psychology of optimal experience (Vol. 41). New York: HarperPerennial.
Deci, E. L. (1971). Effects of externally mediated rewards on intrinsic motivation. Journal of Personality and Social Psychology, 18, 105-155.
Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological review, 95(2), 256.
Dweck, C. (2006). Mindset: The new psychology of success. Random House LLC.
Gopnik, A. (2012). Scientific thinking in young children: Theoretical advances, empirical research, and policy implications. Science, 337(6102), 1623-1627.
Kruglanski, A. W., Friedman, I., & Zeevi, G. (1971). The effects of extrinsic incentive on some qualitative aspects of task performance1. Journal of Personality, 39(4), 606-617.
Lepper, M. R., Greene, D., & Nisbett, R. E. (1973). Undermining children’s intrinsic interest with extrinsic rewards: a test of the “overjustification” hypothesis. Journal of Personality and Social Psychology, 28, 129-137.
Welcome to the DILab blog. This is a blog for members of Georgia Institute of Technology’s Design & Intelligence Laboratory to share their musings as “public intellectuals.” As its name suggests, DILab conducts research at the union of design and intelligence, in the sense of designing intelligent systems as well as intelligent system design. As of today, we are preparing an online course on knowledge-based AI, building intelligent agents that can learn by imitating humans, publishing a paper on fractal reasoning in visual analogy in the Artificial Intelligence Journal, consolidating a partnership with Biomimicry 3.8 on biologically inspired design, launching new collaborations on discovery informatics and computational thinking with the Smithsonian Institution and Vanderbilt University respectively, and getting ready to host the Third Conference on Advances in Cognitive Systems, among many other things. The blog posts too may cover a wide variety of topics and themes. As usual, the blog posts represent the individuals posting them, and do not necessarily reflect the opinions of DILab or Georgia Tech.