Building Intelligent Conversational Tutors and Mentors for Team Collaborative Problem Solving: Guidance from the 2015 Program for International Student Assessment

This chapter describes how conversational computer agents have been used in collaborative problem-solving environments. These agent-based systems are designed to (a) assess the students’ knowledge, skills, actions, and various other psychological states on the basis of the students’ actions and the conversational interactions, (b) generate discourse moves that are sensitive to the psychological states and the problem states, and (c) advance a solution to the problem. We describe how this was accomplished in the Programme for International Student Assessment (PISA) for Collaborative Problem Solving (CPS) in 2015. In the PISA CPS 2015 assessment, a single human test taker (15-year-old student) interacts with one, two, or three agents that stage a series of assessment episodes. This chapter proposes that this PISA framework could be extended to accommodate more open-ended natural language interaction for those languages that have developed technologies for automated computational linguistics and discourse. Two examples support this suggestion, with associated relevant empirical support. First, there is AutoTutor , an agent that collaboratively helps the student answer difficult questions and solve problems. Second, there is CPS in the context of a multi-party simulation called Land Science in which the system tracks progress and knowledge states of small groups of 3–4 students. Human mentors or computer agents prompt them to perform actions and exchange open-ended chat in a collaborative learning and problem-solving environment.