14 research outputs found
What Characterizes a Better Demonstration for Cognitive Modeling by Demonstration?
No full textA simulated student is a machine learning agent that learns a set of cognitive skills by observing solutions demonstrated
by human experts. The learned cognitive skills are converted into a cognitive model for a Cognitive Tutor that
is a computerized tutor that teaches human students the cognitive skills. In this paper, we analyze the characteristics
of the effective demonstrations that lead to quicker and more accurate learning. Results from empirical studies show
that expressive demonstrations (as opposed to abbreviated demonstrations that involve implicit mental operations)
are better for both speed and accuracy of learning. We also found that providing multiple demonstrations of the
same cognitive skill with differing surface features accelerates learning. These findings imply that the ordering of
training sequence as well as the level of detail in demonstration determines the efficiency with which a simulated
student generates a cognitive model
Applying Machine Learning to Cognitive Modeling for Cognitive Tutors
No full textThe aim of this study is to build an intelligent authoring environment for Cognitive Tutors in
which the author need not manually write a cognitive model. Writing a cognitive model usually requires
days of programming and testing even for a well-trained cognitive scientist. To achieve our goal, we have
built a machine learning agent – called a Simulated Student – that automatically generates a cognitive
model from sample solutions demonstrated by the human domain expert (i.e., the author). This paper
studies the effectiveness and generality of the Simulated Student. The major findings include (1) that the
order of training problems does not affect a quality of the cognitive model at the end of the training
session, (2) that ambiguities in the interpretation of demonstrations might hinder machine learning, and
(3) that more detailed demonstration can both avoid difficulties with ambiguity and prevent search
complexity from growing to impractical levels
Creating Cognitive Tutors for Collaborative Learning: Steps Toward Realization
No full textOur long-term research goal is to provide cognitive tutoring of collaboration within a
collaborative software environment. This is a challenging goal, as intelligent tutors have traditionally
focused on cognitive skills, rather than on the skills necessary to collaborate successfully. In this paper, we
describe progress we have made toward this goal. Our first step was to devise a process known as
bootstrapping novice data (BND), in which student problem-solving actions are collected and used to
begin the development of a tutor. Next, we implemented BND by integrating a collaborative software
tool, Cool Modes, with software designed to develop cognitive tutors (i.e., the Cognitive Tutor Authoring
Tools, or CTAT). Our initial implementation of BND provides a means to directly capture data as a
foundation for a collaboration tutor but does not yet fully support tutoring. Our next step was to perform
two exploratory studies in which dyads of students used our integrated BND software to collaborate in
solving modelling tasks. The data collected from these studies led us to identify five dimensions of
collaborative and problem-solving behavior that point to the need for abstraction of student actions to
better recognize, analyze, and provide feedback on collaboration. We also interviewed a domain expert
who provided evidence for the advantage of bootstrapping over manual creation of a collaboration tutor.
We discuss plans to use these analyses to inform and extend our tools so that we can eventually reach our
goal of tutoring collaboration
Rapid Authoring of Intellegent Tutors for Real-World and Experimental Use
No full textAuthoring tools for Intelligent Tutoring Systems are especially valuable if they not only provide a rich set of
options for the efficient authoring of tutoring systems but also support controlled experiments in which the
added educational value of new tutor features is evaluated. The Cognitive Tutor Authoring Tools (CTAT) provide both. Using CTAT, real-world ”Example-Tracing Tutors” can be created without
programming. CTAT also provides various kinds of support for controlled experiments, such as administration of different experimental treatments,
logging, and data analysis. We present two case studies in which Example-Tracing Tutors created with CTAT were used in classroom experiments. The case
studies illustrate a number of new features in CTAT: Use of Macromedia Flash MX 2004 for creating tutor
interfaces, extensions to the Example-Tracing Engine that allow for more flexible tutors, a Mass Production
facility for more efficient template-based authoring, and support for controlled experiments
Example-Tracing Tutors: A New Paradigm for Intelligent Tutoring Systems
No full textThe Cognitive Tutor Authoring Tools (CTAT) support creation of a novel type of tutors called example-tracing tutors. Unlike other types of ITSs (e.g., model-tracing tutors, constraint-based tutors), example-tracing tutors evaluate student behavior by flexibly comparing it against generalized examples of problem-solving behavior. Example-tracing tutors are capable of sophisticated tutoring behaviors; they provide step-by-step guidance on complex problems while recognizing multiple student strategies and (where needed) maintaining multiple interpretations of student behavior. They therefore go well beyond VanLehn’s (2006) minimum criterion for ITS status, namely, that the system has an inner loop (i.e., provides within-problem guidance, not just end-of-problem feedback). Using CTAT, example-tracing tutors can be created without programming. An author creates a tutor interface through drag-and-drop techniques, and then demonstrates the problem-solving behaviors to be tutored. These behaviors are recorded in a “behavior graph,” which can be easily edited and generalized. Compared to other approaches to programming by demonstration for ITS development, CTAT implements a simpler method (no machine learning is used) that is currently more pragmatic and proven for widespread, real-world use by non-programmers. Development time estimates from a large number of real-world ITS projects that have used CTAT suggest that example-tracing tutors reduce development cost by a factor of 4 to 8, compared to “historical” estimates of ITS development time and cost. The main contributions of the work are a novel ITS technology, based on the use of generalized behavioral examples to guide students in problem-solving exercises, as well as a suite of mature and robust tools for efficiently building real-world ITSs without programming.</p
Collaboration and Cognitive Tutoring: Integration, Empirical Results, and Future Directions
No full textIn this paper, we describe progress we have made toward providing
cognitive tutoring to students within a collaborative software environment. First, we
have integrated a collaborative software tool, Cool Modes, with software designed to
develop Cognitive Tutors (the Cognitive Tutor Authoring Tool). Our initial
integration provides a means to capture data that acts as the foundation of a tutor for
collaboration but does not yet fully support actual tutoring. Second, we've performed
two exploratory studies in which dyads of students used our software to collaborate in
solving modelling tasks. These studies uncovered five dimensions of observed
behavior that point to the need for abstraction of student actions to better recognize,
analyze, and correct collaborative steps in problem solving. We discuss plans to
incorporate such analyses into our approach and to extend our tools to eventually
provide tutoring of collaboration
Cognitive Tutoring of Collaboration: Developmental and Empirical Steps Towards Realization
No full textIn this paper, we describe developmental and empirical steps we have taken toward
providing Cognitive Tutoring to students within a collaborative software environment. We have
taken two important steps toward realizing this goal. First, we have integrated a collaborative
software tool, Cool Modes, with software designed to develop Cognitive Tutors (the Cognitive
Tutor Authoring Tool). Our initial integration does not provide tutoring per se but rather acts as a
means to capture data that provides the beginnings of a tutor for collaboration. Second, we have
performed an initial study in which dyads of students used our software to collaborate in solving a
classification / composition problem. This study uncovered five dimensions of analysis that our
approach must use to help us better understand student collaborative behavior and lead to the
eventual development of a Cognitive Tutor for collaboration. We discuss our plans to incorporate
such analysis into our approach and to run further studies
The Cognitive Tutor Authoring Tools (CTAT): Preliminary Evaluation of Efficiency Gains
No full textIntelligent Tutoring Systems have been shown to be effective in a
number of domains, but they remain hard to build, with estimates of 200-300
hours of development per hour of instruction. Two goals of the Cognitive Tutor
Authoring Tools (CTAT) project are to (a) make tutor development more
efficient for both programmers and non-programmers and (b) produce scientific
evidence indicating which tool features lead to improved efficiency. CTAT
supports development of two types of tutors, Cognitive Tutors and Example-
Tracing Tutors, which represent different trade-offs in terms of ease of
authoring and generality. In preliminary small-scale controlled experiments
involving basic Cognitive Tutor development tasks, we found efficiency gains
due to CTAT of 1.4 to 2 times faster. We expect that continued development of
CTAT, informed by repeated evaluations involving increasingly complex
authoring tasks, will lead to further efficiency gains
Rapid Authoring of Intelligent Tutors for Real-World and Experimental Use
No full textAuthoring tools for Intelligent Tutoring Systems are
especially valuable if they not only provide a rich set of
options for the efficient authoring of tutoring systems
but also support controlled experiments in which the
added educational value of new tutor features is
evaluated. The Cognitive Tutor Authoring Tools
(CTAT) provide both. Using CTAT, real-world
”Example-Tracing Tutors” can be created without
programming. CTAT also provides various kinds of
support for controlled experiments, such as
administration of different experimental treatments,
logging, and data analysis. We present two case
studies in which Example-Tracing Tutors created with
CTAT were used in classroom experiments. The case
studies illustrate a number of new features in CTAT:
Use of Macromedia Flash MX 2004 for creating tutor
interfaces, extensions to the Example-Tracing Engine
that allow for more flexible tutors, a Mass Production
facility for more efficient template-based authoring,
and support for controlled experiments
