Ernst Strüngmann Forum

The central topic of the 26th Forum—interactive task learning—was brought to our attention by Kevin A. Gluck and John E. Laird. The overarching idea was to convene a diverse group of experts to examine the processes by which new tasks are acquired through natural interaction between humans, humans and agents, and agents. It was felt that the fractionated state of relevant scientific and technical disciplines had thus far hindered progress in this area—one fundamental to both artificial intelligence and cognitive psychology. Thus, creating collaboration within the broad research community as well as delineating research challenges and future trajectories were set as primary goals.

Joining us on the Program Advisory Committee to realize these goals were Ken Ford, Elena Lieven, Luc Steels, and Niels Taatgen. From April 22–24, 2016, the committee met to refine the scientific framework of the proposal and identify participants to the Forum, which was held in Frankfurt am Main from May 21–26, 2017.

This volume synthesizes the ideas and perspectives that emerged from the entire process and is comprised of two types of contributions:

Key questions and aspects of interactive task learning are presented. Most of these chapters served as background to the Forum while others emerged out of the discussion. Both have been peer reviewed and edited to provide up-to-date information.

In Chapters 3, 7, 11, and 15, the working groups from the Forum provide a synthesis of their multifaceted discussions. Edited to ensure accessibility, these chapters should not be understood as proceedings or consensus documents. Their intent is to summarize perspectives, expose diverging opinions as well as remaining open questions, and highlight areas for future enquiry.

Every Forum creates its own unique dynamics and puts demands on all who participate. Each invitee played an active role at this Forum and for their efforts, I wish to thank everyone. I extend a special word of appreciation to the Program Advisory Committee, to the authors and reviewers of the background papers, as well as to the moderators of the individual working groups: Elena Lieven, Niels Taatgen, John Laird, and Kevin Gluck. For their efforts in drafting and finalizing the reports, special recognition goes to the rapporteurs of the working groups: Andrea Thomaz, Robert Wray III, Dario Salvucci, and Julie Shah. Finally, I wish to extend my sincere appreciation to Kevin Gluck and John Laird, whose commitment and cooperation were integral at each stage.

To conduct its work, the Ernst Strüngmann Forum relies on institutional stability and an environment that encourages free thought. The generous support of the Ernst Strüngmann Foundation, established by Dr. Andreas and Dr. Thomas Strüngmann in honor of their father, enables the Ernst Strüngmann Forum to pursue its work in the service of science. Additional partnerships include the Scientific Advisory Board, which ensures the scientific independence of the Forum; the German Science Foundation, for its supplemental financial support; and the Frankfurt Institute for Advanced Studies, which shares its vibrant setting with the Forum.

Breaking new intellectual ground is never easy. Yet, when the edges of the unknown begin to appear and the gaps in understanding are identified, the act of formulating strategies to fill these gaps becomes a most invigorating activity. On behalf of everyone involved, I hope this volume will spur further discussion and research into the field of interactive task learning.

Human learning has been the subject of extensive research in multiple areas of science. People are always learning, from whatever sources of knowledge are available. Language and other forms of natural communication enable us to master novel tasks quickly; once we do, we often share the resultant knowledge with others. In just a few minutes, we can grasp how to play a new game, use a new device (e.g., smart phones, industrial machinery), or assist a disabled family member in meeting specific challenges. Importantly, as we learn and hone performance on a task, we adapt in real time to emergent needs—sometimes figuring things out for ourselves, sometimes interacting with others to gain efficiencies or address any problems we encounter.

Contemporary artificial agents, by contrast, are bound to the specific tasks for which they were originally programmed. Even systems designed to acquire knowledge and expertise can learn only a single task at a time (e.g., Chess, Go, video games), becoming idiot savants with amazing capabilities, but without any abilities beyond that narrow specialization. Without doubt, advances in artificial intelligence, cognitive science, and robotics point to future systems with sufficient cognitive and physical capabilities to perform a wide variety of diverse tasks. But how will they learn tasks that arise unexpectedly—tasks that cannot be anticipated and therefore preprogrammed or trained for? How can agents pursue a task when there is insufficient prior knowledge or time for exploration to guide learning?

Interactive task learning (ITL) attempts to answer those questions by providing a conceptual framework for agents to learn not only how to perform tasks better, but also to learn new tasks from scratch through natural, real-time interactions with others. ITL involves interactions between an agent (human or machine), its world, and, crucially, other agents in the world. ITL is a bidirectional process between teacher and learner (both of which can be humans or machines) that results in collaboration and knowledge creation.

The catalyzing idea behind ITL is as follows: for artificial systems to learn from and teach us entirely new things at any given moment, we must advance beyond the traditional approach of creating specialized AI agents for single, predetermined niche purposes and instead incorporate a rich set of natural interaction and learning mechanisms into our systems. This involves two crucial requisites. First, the way in which artificial systems learn and teach new tasks must be natural for people, not constrained by traditional programming and digital forensics. Second, learning and performing multiple tasks can only be bounded by physical and informational limits, not by design, implementation, or optimization for single task performance.

The concept of ITL is controversial, in that it disrupts the status quo and involves diverse challenges. ITL requires theoretical and practical advances in the integration of a broad range of capabilities associated with cognition, including extracting task-relevant meaning from perception, task-relevant action, grounded language processing, dialogue and interaction management, integrated knowledge-rich relational reasoning, problem solving, learning, and metacognition. This integration contrasts with the general trend toward increasing fragmentation and focus on narrow capabilities and problems. Beyond these challenges, ITL creates an opportunity to rethink the fundamental nature of our most advanced and capable artifacts. How can we move beyond artifacts that are designed for a single use or purpose, to ones that can be dynamically adapted to our changing needs, increasing the rate of our progress and the quality of our lives as individuals and societies? Isolated efforts to develop more intelligent agents and robots are already underway in areas such as healthcare, in-home assistance, education, and transportation. We propose the missing link among them is the unifying vision of ITL.

Our optimism that the time is right for a coordinated and concerted push toward ITL is grounded in our assessment that despite shortcomings, gaps, and challenges, the research community has made progress on important component capabilities and their integration. To shape R&D investments in a way that advances ITL in artificial systems requires the identification of broad organizing themes. Pace, persistence, and partnering are core characteristics that constitute research challenges around which we can rally our science and technology investments.

Pace, Persistence, and Partnering

The human capacity for rapid, nearly instantaneous learning of entirely new tasks on the basis of brief communications and one, two, or a few demonstrations sets a pace requirement for ITL. The pace of the interaction, the pace of the teaching and learning, and the pace of task completion must all occur on timescales aligned with and amenable to real-time human experience.

Humans are engines of creation. From the imaginative play of early childhood, to the generative nature of language, to scientific discovery and technological innovation, we are constantly creating new constructs, concepts, and capabilities. Our persistence throughout these activities requires us to both assimilate and accommodate newly gained knowledge and existing knowledge. As we work to create intelligent artifacts that interact, we must recognize that it will never be possible to anticipate and represent, in advance, all the knowledge and skill that may be required in the future. ITL agents must be able to learn and adapt continually with robust success, over a long period of time in environments that are dynamic, nonstationary, and boundlessly novel.

Human beings help each other. It is what we do. We organize in ways that support joint objectives and goals. Our most valued relationships are with family, friends, partners, and teammates. These relationships develop over time out of shared experiences in which we demonstrate an ability and willingness to be there for each other in times of need. We are at our best when we take the initiative to assist or compensate without being asked, simply because we know it will be helpful. By contrast, contemporary machine artifacts do none of this. They function merely as tools, responding as designed, reactive but not proactive. They are unable to engage in true partnering. ITL systems need to be more like partners or teammates, and not merely tools.

Each of these core characteristics has received some attention from isolated subsets of the research community. To achieve ITL in future agents, we must find a way to integrate these characteristics into systems. This will not be easy, for at the core of each characteristic and their integration is the challenge of understanding.

The Challenge of Understanding

Perhaps the most important limitation of our contemporary intelligent machines is that they are not capable of understanding with the depth and breadth found in humans. Many impressive accomplishments have been achieved in the cognitive and computational sciences in recent decades. Most of those are best known to isolated subcommunities of researchers toiling away on issues with great scientific merit. A precious few have captured the imagination of the public due to high profile events, demonstrations, and competitions. Algorithmic advances, blazing fast processors, and massive amounts of training data make it possible to show that silicon-based computation can classify objects, learn well-defined games, and answer some types of questions as well as or better than people can. Less well hyped is the characteristic fragility of these systems. When they are wrong, they are often wrong in ways that are surprising and confusing to people. This is because people understand the questions, images, and activities within the broad context of not just a single task, but within the myriad of tasks, experiences, and relationships they develop over time in ways that the algorithms do not.

At least as troubling as the lack of understanding in our most advanced artificially intelligent machine learners is the fact that we often don’t understand them. This is certainly true for the general public, who tend to ascribe assortments of sophisticated humanlike intellectual capacities to computational systems where it is not warranted. It is often also true for the developers of some of our most impressive learning machines. Among those working at the leading edge of science and technology, the issue is not one of unjustified anthropomorphism. Rather, it is the reality of human cognitive limitations running up against complex, hybrid computational systems. The emphasis on powerful learning mechanisms scaled for use on big data sources has abandoned transparency and left even the innovators of these capabilities scratching their heads and asking, “Why and how is it doing that? What did it learn?” Generally those questions can be answered by engaging in some committed digital forensics, but the time and energy required for those analyses far exceed what would be tolerable in the context of ITL. Queryability, explainability, and transparency must be baked into these systems in order to foster natural, efficient understanding.

Finally, in a recursive descent into scientific and technological challenges, as a research community we must face the reality that the root cause of our machines’ poor understanding and of our poor understanding of complex learning machines is the fact that we simply do not understand the concept of understanding. There is, in effect, no scientific consensus about what understanding actually is, despite an abundance of work by philosophers, psychologists, neuroscientists, and computer scientists. Indicative of this absence of agreement is a great deal of ambiguity regarding how to assess understanding. This should come as no surprise, given the inconsistent and haphazard manner in which we, as individuals, evaluate the understanding of other people in our daily lives. We tend to assume a great deal of understanding in the minds of others. Sometimes those assumptions are valid and supported by social cues, prior experience, or knowledge of the other, which makes these assumptions defensible. Other times they are simply efficient conveniences. Rarely do we bother to rigorously evaluate the extent to which another person understands.

Up to now, we have been able to overlook our ignorance regarding the fundamental nature of understanding, our poor understanding of complex learning systems, and the absence of understanding in our machines. We have been satisfied with the traditional approach of implementing systems for pre-determined niche purposes. However, the vision for ITL in artificial systems creates a forcing function to address these issues. The development, improvement, and evaluation of understanding in humans, robots, and agents is critical to the creation of ITL.

Moving Forward: A Multidisciplinary Challenge

Clearly, we are enthusiastic about the potential societal benefits that ITL systems could bring. Nonetheless, we appreciate that the challenges are daunting. To even begin, experts from multiple areas of science and technology must be able to communicate, find common ground, and implement novel capabilities across disciplinary divides.

With the support of the Ernst Strüngmann Forum we sought to initiate a dialogue among experts from robotics, cognitive modeling, computer science, artificial intelligence, and developmental and comparative psychology. This discourse aimed to analyze how humans and artificial agents acquire new tasks through natural interactions as well as to define ITL from various perspectives, in an effort to establish a foundational reference and organizing framework. The results of this multifaceted dialogue are captured in this volume. Organized around the following primary topics, each contribution explores key aspects of ITL:

1. Knowledge: In Chapter 3, Robert Wray III et al. discuss the functional roles of knowledge in ITL, examine central challenges that must be overcome, and pose research questions to direct future research. From a formal, computational perspective, Christian Lebiere (Chapter 4) presents different forms of knowledge and skills involved in ITL. Through an examination of the collaborative interactions inherent in learning and teaching, Charles Rich (Chapter 5) analyzes the abstract form, nature, and organization of task knowledge. Concluding this section, Niels Taatgen (Chapter 6) explores what is needed to construct a cognitive architecture capable of supporting flexible knowledge and skills.

2. Interaction: In Chapter 7, Andrea Thomaz et al. consider which qualities of human interaction and learning will be most effective and natural to incorporate into an ITL agent; central to this is the alignment of common ground between a teacher agent and a learner agent. In his analysis of natural forms of purposeful interaction among humans, Stephen Levinson (Chapter 8) delineates the basic organization of interactive language use and discusses the challenge of incorporating the predictive nature of human comprehension into an ITL agent. In Chapter 9, Joyce Chai et al. outline the different types of knowledge that can be transferred between agents and discuss the perception, action, and coordination capabilities that enable teaching–learning interactions; in addition, they consider challenges and research opportunities associated with enabling natural interaction in artificial agents. To conclude this section, Wayne Gray et al. (Chapter 10) explore how experimental psychology, machine learning, and advanced statistical analyses can be used to understand the complexity of interactive performance in complex tasks involving single or multiple interactive agents in dynamic environments.

3. Instruction: In Chapter 11, Julie Shah et al. present frameworks, models, and methods for task instruction, broadly connecting structural and adaptive improvements to instruction, historical developments in programming, and the extraordinary challenge that fluid, flexible, co-constructive task instruction and learning places on the vision for ITL. In Chapter 12, Kurt VanLehn looks at prototypical human tutoring behavior, analyzing what exceptional tutors sometimes do (but most tutors do not) and comparing the effectiveness of human versus computer tutors. In Chapter 13, Katrin Beuls et al. examine what type of general architecture is needed to construct artificial agents that can assume the role of teacher (by carrying out teaching strategies) or the role of learner (by carrying out learning strategies that benefit from these teaching strategies); they argue that a meta-layer is necessary to understand and implement strategies and point to operational examples in the domain of second language teaching. In Chapter 14, Arthur Still et al. explore the concept of creativity and its relationship to the development of education theory, focusing on what is necessary to inform teaching practice and development of education technology.

4. Learning: Summarizing their discussions at the Forum, Dario Salvucci et al. explore in Chapter 15 the learning of task knowledge through interaction, the capabilities that facilitate learning, aspects of interaction that relate closely to learning, as well as evaluation dimensions and metrics for ITL systems. Based on knowledge of preexisting capabilities that appear early in human development, Franklin Chang (Chapter 16) introduces a world-state prediction model—one that can learn detailed physical regularities in the environment and develop representations for predicting the actions and goals of animate agents—to suggest that prediction and prediction error are capabilities that could improve ITL systems. In Chapter 17, using an existing agent, Rosie, to illustrate how an ITL agent can learn many tasks in a variety of domains, John Laird et al. present characteristics of the learning problem and examine how these influence underlying learning algorithms; learning approaches are discussed that respond to the unique challenges of ITL.

Throughout this process, the open exchange of ideas and perspectives—hallmarks of the Ernst Strüngmann Forum—was bolstered by our own inclination toward asking lots of questions, especially the hard ones, and to accept disagreement, countervailing opinions, and inevitable failures on the path of progress. As one might imagine, many questions surfaced and, where appropriate, ways of pursuing these have been highlighted. Two priorities that emerged, however, have been given special attention. The first, a common reference frame to guide future discussion, is presented by Tom Mitchell et al. in Chapter 2. The second is an appreciation that we must commence with ethical considerations now, even as we debate the nature, viability, and path toward ITL. There will be valid security and privacy concerns, and it is certain that people with malicious intent will attempt to repurpose ITL for harm. Thus, now is the time to think about and take action on these matters. To that end, in Chapter 18, Matthias Scheutz examines different ethical aspects of ITL.

Despite all the challenges, we believe ITL offers great potential for humanity and hope this volume will inspire the international research community to pursue the necessary science and technology. We look forward to working with a global community of researchers to realize this vision.

Acknowledgments

We thank the Ernst Strüngmann Foundation for its extraordinary commitment to the exploration of multidisciplinary scientific challenges, and especially for approving and funding this Forum on Interactive Task Learning. However, organizations are only as good as the people within them, and it is the collective contributions of the individuals leading the Ernst Strungmann Forum that make it the world-class, impactful experience that it is. Our sincere appreciation to Julia Lupp, its Director, for her deep immersion and commitment, unwavering support, and impressive patience, and to Aimée Ducey-Gessner, Marina Turner, and Catherine Stephen for their excellent professional support throughout this process.

Participating in an Ernst Strungmann Forum is a significant investment of time and intellectual energy. We thank all of the participants for setting aside their many other existing commitments to join us in this endeavor. Special appreciation is due to our Program Advisory Committee (Ken Ford, Elena Lieven, Julia Lupp, Luc Steels, and Niels Taatgen), who worked with us to shape new ideas and rough intentions into a more complete and concrete plan for the Forum. We must also recognize the impressive work of our rapporteurs (Dario Salvucci, Julie Shah, Andrea Thomaz, and Bob Wray) who toiled diligently throughout and following the Forum to summarize, represent, and organize diverse discussion points into the group reports introducing each section of the book.

Finally, although we were as comprehensive and inclusive as possible, there is no way to include everyone who is doing important and relevant work in a single event such as this. Thus we thank our many additional colleagues and collaborators from the cognitive, computing, social, and psychological sciences who are chipping away at the barriers, making progress on the challenges, and choosing to travel the path toward ITL. You are inspiring, and we look forward to learning with you in future interactions.

PDF of the Chapter

Abstract
To support a precise discussion of interactive task learning, the problem setting in which teachers and learners interact in a shared world must be clearly defined and understood. This chapter provides a formalism to enable discussion of the different types of interactive learning: from teaching a robot to grasp a novel object, to instructing a mobile phone how to reach a friend in an emergency. It provides a way to speak precisely about notions such as shared knowledge between teachers and learners, presents working definitions of the internal structures of the agent, and describes the relationships between the task environment and the communication channel. It focuses on the problem of interactive task learning, not its solution, as a backdrop to further discourse in this volume.

PDF of the Chapter

Abstract

What knowledge needs to be learned to acquire a novel task? What background knowledge does an agent need to use newly acquired knowledge effectively? This chapter considers the functional roles of knowledge in task learning. These roles of knowledge span interaction with other entities and the environment and core functional capabilities of the reasoning system itself (i.e., architecture). Perspectives are offered on the definition of “task” and the relationship between task and knowledge. In addition, three specific challenges central to the role of knowledge in interactive task learning (ITL) are examined: the identification of architectural primitives (basic functional and representational building blocks) needed for ITL, requirements for enabling shared understanding (“common ground”) between learner and instructor, and conditions that support projection and anticipation of future states. In conclusion, specific research questions are put forth to address these challenges and advance ITL as a field of inquiry.

PDF of the Chapter

Abstract
Computational models offer a precise, quantitative way to represent the cognitive processes and representations involved when an agent interacts with another agent: from the receiving of instructions, to their interpretation, to the processes involved in learning to perform a task. This chapter discusses various forms of knowledge and skills involved in interactive task learning (ITL). It describes the components and processes in cognitive architectures relevant to ITL, organized around dichotomies of declarative knowledge and procedural skills, symbolic representations and subsymbolic statistics, as well as cognitive, perceptual, and motor processes. One specific cognitive architecture, ACT-R, serves to focus discussion. Using a model of interactive learning in decision making, it demonstrates how these components and processes interact. Representation, learning, and processing issues are discussed both in isolation as well as in the context of this integrated task learning model.

PDF of the Chapter

Abstract
Learning and teaching are best viewed as a collaborative interaction. As participants, both the teacher and learner share the goal of increasing the learner’s abilities. Yet what does it mean to know how to do something? This chapter analyzes the abstract form, nature, and organization of task knowledge and illustrates these concepts using a shared task of tire rotation. It applies a hierarchical decomposition of knowledge for interactive task learning that involves three levels: domain knowledge, procedural knowledge, and metaknowledge. In addition, the traditional distinction between symbolic versus nonsymbolic task knowledge is noted. Representative examples are given, and open questions and unresolved problems are highlighted as suggested directions for future inquiry.

PDF of the Chapter

Abstract
Interactive task learning requires knowledge and skills that are highly flexible and composable, and a cognitive architecture to support this. Cognitive architectures aim to bridge the gap between the brain and intelligence, providing a formal level of description for rigorous theories of behavior. Architectures typically operate on a single level of abstraction, but this may be too limited for interactive task learning. Instead, architectures with multiple levels of abstraction should be considered, each with their own formalisms and learning mechanisms. Each level should be able to explain the abstraction level above it, thus creating a reductionist hierarchy of theories to model human intelligence, not with a single formalism, but with several.

PDF of the Chapter

Abstract

This chapter considers the qualities of human interaction and learning that will be most effective and natural to incorporate into any interactive task learning agent, and focuses specifically on the interactions involved in learning from explicit instruction. At the center of this interaction is a process that brings the common ground between a teacher agent and a learner agent into alignment. Errors or misalignments to this common ground drive the interactive learning process. The importance of timing is highlighted as is the dynamics of an interaction, as a communication channel itself, in this alignment process.

PDF of the Chapter

Abstract
The design of an interactive robot should make crucial reference to the observed properties of human interaction. Obviously, human communicative interaction varies across languages and cultures, but remarkably uniform is the basic organization of interactive language use: participants take short turns at talking while avoiding overlap; they utilize a basic inventory of action–response pairs (e.g., question–answer), which can be recursively employed; they have systematic backup systems for communicative difficulties and deploy multimodal signals (speech, gesture, facial expression, gaze) to disambiguate or reinforce intended content. This chapter spells out these design properties and makes the point that human comprehension is fundamentally predictive, and has to be so to achieve the typically rapid response times despite the large latencies involved in generating speech. These properties may pose a substantial, even insuperable, hurdle for a fully humanoid interactive robot, but fortunately humans are excellent at adapting to interactants with restricted capabilities, such as children, foreigners, or aphasics.

PDF of the Chapter

Abstract

This chapter focuses on the main challenges and research opportunities in enabling natural interaction to support interactive task learning. Interaction is an exchange of communicative actions between a teacher and a learner. Natural interaction is viewed as an interaction between a human and an agent that leverages ways in which humans naturally communicate and does not require prior expertise. The goal of communication is to achieve common ground and allow the learner to acquire new task knowledge. This chapter outlines the different types of knowledge that can be transferred between agents and discusses the perception, action, and coordination capabilities that enable teaching–learning interactions.

PDF of the Chapter

Abstract
Studying the essence of interaction requires task environments in which changes may arise due to the nature of the environment or the actions of agents in that environment. In dynamic environments, the agent’s choice to do nothing does not stop the task environment from changing. Likewise, making a decision in such environments does not mean that the best decision, based on current information, will remain “best” as the task environment changes. This chapter summarizes work in progress which brings the tools of experimental psychology, machine learning, and advanced statistical analyses to bear on understanding the complexity of interactive performance in complex tasks involving single or multiple interactive agents in dynamic environments.

PDF of the Chapter

Abstract
An early concept of interactive task learning (ITL) assumed a human teacher and machine learner. This book broadens the thinking about this relationship by explicitly allowing flexibility regarding the teacher and learner roles. Future ITL systems will be maximally useful and beneficial to the extent that they are effective and efficient learners as well as effective and efficient instructors. Focusing on task instruction, the primary goal of this chapter is to relate the critical role of instruction in ITL to key existing literature from related areas of research. The general concept of co-constructive task instruction is introduced and differentiated from traditional conceptualizations of fixed instructor and learner roles. Frameworks, models, and methods for task instruction are discussed, and broad connections are made between ITL and structural and adaptive improvements to instruction, historical developments in programming, and the extraordinary challenge that fluid, flexible, co-constructive task instruction and learning places on the vision for ITL.

PDF of the Chapter

Abstract
People teaching an agent or robot might use the same methods that they use when tutoring a human student. Because teaching agents and robots is a central topic of this Ernst Strüngmann Forum, this chapter reviews research that characterizes human tutoring. Most of this research was done to improve the design of computer-based tutoring systems, which were assumed to be inferior to human tutors. However, it turns out that human tutors and a certain class of tutoring systems actually behave quite similarly, and their effectiveness is about the same. This chapter begins with a description of prototypical human tutoring behavior before discussing some common hypotheses about human tutoring behavior, which turn out to be unsupported by studies. It concludes with an attempt to synthesize these descriptions and apply them to the goals set forth at this Forum.

PDF of the Chapter

Abstract
A strategy is a way to make decisions that come up when handling a task. It requires a problem solver able to address routine cases and a set of diagnostics and repairs to handle, in a flexible way, unusual or unforeseen situations. Between humans, interactive task learning and teaching appear to involve strategies at three levels: (a) the execution of a task with available knowledge (task strategy), (b) interactive learning to expand the available knowledge and thus become a better problem solver in the future (learning strategy), and (c) interactive teaching or tutoring to help others learn (teaching strategy). This chapter examines the general architecture that is needed to build artificial agents that can play either the role of teacher, by carrying out teaching strategies, or the role of learner, by carrying out learning strategies that benefit from these teaching strategies. Focus is on artificial teachers that interact with humans or artificial learners as well as on artificial learners that interact with human or artificial teachers. We argue that the use of a meta-layer is of primary importance for understanding and implementing strategies and point to operational examples from an implementation of this hypothesis in the domain of second-language teaching.

PDF of the Chapter

Abstract
This chapter provides a historical perspective on the concept of creativity and its relationship to the development of education theory during the first half of the twentieth century. In the early twentieth century, creativity had a very specific meaning, which expanded in the mid- to late twentieth century into a more general, and in our view less useful, meaning. These two perspectives are linked to two conflicting educational theories, represented by Edward Lee Thorndike and John Dewey. Dewey described learning as a natural part of being an inquiring human being in a social and physical world, whereas Thorndike’s view was more reductionist, based on stimulus–response connections. The Thorndike’s theory gained prominence and still dominates today, over the Deweyan theory, due in part to the ease with which it can be experimentally tested.

Ideas are developed into a two-part manifesto to inform teaching practice and the development of education technology. The first part delineates the conditions for creative feedback in social learning and encapsulates a Deweyan educational approach. The second part describes the characteristics of education technology that can be used to experiment with creative feedback and social learning, and establishes how we can begin to validate experimentally the Deweyan theory of education.

Interactive task learning considers the challenge of interactively training bots to carry out a task. This chapter is most relevant to medium-term and future tasks for bots within a social context involving humans and bots, and may offer subjective or dynamic evaluation criteria. Bot instructors working with these types of tasks may benefit from considering the complexity and nuances of creative feedback.

PDF of the Chapter

Abstract
How does an agent acquire (i.e., learn) knowledge and information about a specific task by interacting with a teacher, so that ultimately the agent is able to execute the task successfully? This chapter reviews critical aspects of the learning process in interactive task learning (ITL). It discusses learning task knowledge through interaction, capabilities that facilitate learning, aspects of interaction that relate closely to learning, and evaluation dimensions and metrics for ITL systems. Given the interconnected nature of ITL, it also explores relationships between learning, knowledge, interaction, and tasks: how tasks influence learning, how knowledge should be represented, and what types of information and communication are needed to facilitate learning.

PDF of the Chapter

Abstract
Humans are better than artificial computational systems at learning to do new tasks through interaction. Part of this ability stems from preexisting capabilities that appear early in human development. Children have internal physical models of how objects move and they attribute mental states (e.g., goals, beliefs) to objects when their behavior is unpredictable. They are also able to develop context-specific rules and identify how to help others achieve their goals. To explore how these abilities can be transferred to interactive task learning (ITL) systems, this chapter proposes a world-state prediction model. The prediction model can learn detailed physical regularities in the environment and is able to develop representations for predicting the actions and goals of animate agents. The model suggests that prediction and prediction error are capabilities that could improve ITL systems.

PDF of the Chapter

Abstract
In most learning problems, a single type of task knowledge is learned using a single specialized learning algorithm designed and optimized for that specific type of knowledge and the environment in which it is learned. In contrast, interactive task learning (ITL) involves learning all types of task knowledge where such specialization is impossible. This chapter describes these characteristics of the ITL learning problem, which distinguish it from other learning problems, and examines how those characteristics influence the underlying learning algorithms. Throughout our discussion, the Rosie agent is used as an example of an ITL agent that can learn many tasks in a variety of domains. The distinguishing characteristics explored include learning across different domains, learning diverse task knowledge, interactivity in learning, the situated aspects of learning, and how an ITL agent can exploit multiple data sources. Learning approaches are then discussed that can be used in ITL from the perspective of how they address the unique challenges of ITL.

PDF of the Chapter

Abstract
As with all transformative technologies, humanity needs to analyze the ethical challenges and potential impacts associated with implementation. This chapter explores fundamental questions that pertain to interactive task learning (ITL): What is being taught and what are the associated risks? What are the dynamics of human–machine instruction? What effects will ITL have on human instructors and society? It explores the long-term impact that ITL could have on humans and human society, and discusses concerns valid to both machine learning and ITL (e.g., how to ensure that machines will learn knowledge that they can put to good use, that they will serve humans well and not become deviant). Importantly, it stresses the unique aspects of ITL and proposes that the time to think about and take action on these concerns is now.

PDF of the Chapter