8 research outputs found
Optimizing Student Models for Causality
No full textComplex student models often include key parameters critical to their behavior and effectiveness. For example, one meta-cognitive model of student help-seeking in intelligent tutors includes 15 rules and 10 parameters. We explore
whether or not this model can be improved both in accuracy and generalization by using a variety of techniques to select and tune parameters.We show that such techniques are important by demonstrating that the normal method of fitting parameters on an initial data set generalizes poorly to new test data sets. We then show that
stepwise regression can improve generalization, but at a cost to initial performance. Finally, we show that causal search algorithms can yield simpler models that perform comparably on test data, but without the loss in training set performance. The resulting help-seeking model is easier to understand and classifies a more realistic number of student actions as help-seeking errors
Rapid Authoring of Intelligent Tutors for Real-World and Experimental Use
No full textAuthoring tools for Intelligent Tutoring Systems are
especially valuable if they not only provide a rich set of
options for the efficient authoring of tutoring systems
but also support controlled experiments in which the
added educational value of new tutor features is
evaluated. The Cognitive Tutor Authoring Tools
(CTAT) provide both. Using CTAT, real-world
”Example-Tracing Tutors” can be created without
programming. CTAT also provides various kinds of
support for controlled experiments, such as
administration of different experimental treatments,
logging, and data analysis. We present two case
studies in which Example-Tracing Tutors created with
CTAT were used in classroom experiments. The case
studies illustrate a number of new features in CTAT:
Use of Macromedia Flash MX 2004 for creating tutor
interfaces, extensions to the Example-Tracing Engine
that allow for more flexible tutors, a Mass Production
facility for more efficient template-based authoring,
and support for controlled experiments
Cognitive Tutors as Research Platforms: Extending an Established Tutoring System for Collaborative and Metacognitive Experimentation
No full textCognitive tutors have been shown to increase student learning
in long-term classroom studies but would become even more effective if they
provided collaborative support and metacognitive tutoring. Reconceptualizing
an established tutoring system as a research platform to test different
collaborative and metacognitive interventions would lead to gains in learning
research. In this paper, we define a component-based architecture for
such a platform, drawing from previous theoretical frameworks for tutoring
systems. We then describe two practical implementation challenges not
typically addressed by these frameworks. We detail our efforts to extend a
cognitive tutor and evaluate our progress in terms of flexibility, control,
and practicality
Toward Tutoring Help Seeking; Applying Cognitive Modeling to Meta-Cognitive Skills
No full textThe goal of our research is to investigate whether a Cognitive
Tutor can be made more effective by extending it to help students acquire
help-seeking skills. We present a preliminary model of help-seeking behavior
that will provide the basis for a Help-Seeking Tutor Agent. The model,
implemented by 57 production rules, captures both productive and unproductive
help-seeking behavior. As a first test of the model’s efficacy, we used it
off-line to evaluate students’ help-seeking behavior in an existing data set
of student-tutor interactions, We found that 72% of all student actions represented
unproductive help-seeking behavior. Consistent with some of our
earlier work (Aleven & Koedinger, 2000) we found a proliferation of hint
abuse (e.g., using hints to find answers rather than trying to understand). We
also found that students frequently avoided using help when it was likely to
be of benefit and often acted in a quick, possibly undeliberate manner. Students’
help-seeking behavior accounted for as much variance in their learning
gains as their performance at the cognitive level (i.e., the errors that
they made with the tutor). These findings indicate that the help-seeking
model needs to be adjusted, but they also underscore the importance of the
educational need that the Help-Seeking Tutor Agent aims to address
Toward Meta-Cognitive Tutoring: A Model of Help-Seeking with a Cognitive Tutor
No full textThe research reported in this paper focuses on the hypothesis that an intelligent tutoring system
that provides guidance with respect to students’ meta-cognitive abilities can help them to become better
learners. Our strategy is to extend a Cognitive Tutor (Anderson, Corbett, Koedinger, & Pelletier, 1995) so
that it not only helps students acquire domain-specific skills, but also develop better general help-seeking
strategies. In developing the Help Tutor, we used the same Cognitive Tutor technology at the metacognitive
level that has been proven to be very effective at the cognitive level. A key challenge is to
develop a model of how students should use a Cognitive Tutor’s help facilities. We created a preliminary
model, implemented by 57 production rules that capture both effective and ineffective help-seeking
behavior. As a first test of the model’s efficacy, we used it off-line to evaluate students’ help-seeking
behavior in an existing data set of student-tutor interactions. We then refined the model based on the results
of this analysis. Finally, we conducted a pilot study with the Help Tutor involving four students. During
one session, we saw a statistically significant reduction in students’ meta-cognitive error rate, as
determined by the Help Tutor’s model. These preliminary results inspire confidence as we gear up for a
larger-scale controlled experiment to evaluate whether tutoring on help seeking has a positive effect on
students’ learning outcomes
An Architecture to Combine Meta-Cognitive and Cognitive Tutoring: Pilot Testing the Help Tutor
No full textGiven the important role that meta-cognitive processes play in learning,
intelligent tutoring systems should not only provide domain-specific assistance, but should also aim to help students in acquiring meta-cognitive skills. As a step toward this goal, we have constructed a Help Tutor, aimed at improving students’ help-seeking skill. The Help Tutor is based on a cognitive model of students’ desired help-seeking processes, as they work with a Cognitive Tutor (Aleven et al., 2004). To provide metacognitive
tutoring in conjunction with cognitive tutoring, we designed an architecture in
which the Help Tutor and a Cognitive Tutor function as independent agents, to facilitate re-use of the Help Tutor. Pilot tests with four students showed that students improved their help-seeking behavior significantly while working with the Help Tutor. The improvement could not be attributed to their becoming more familiar with the domain specific
skills being taught by the tutor. Although students reported afterwards that they
welcomed feedback on their help-seeking behavior, they seemed less fond of it when actually advised to act differently while working. We discuss our plans for an experiment to evaluate the impact of the Help Tutor on students’ help-seeking behavior and learning, including future learning, after their work with the Help Tutor
An Architecture to Combine Meta-Cognitive and Cognitive Tutoring: Pilot Testing the Help Tutor
No full textGiven the important role that meta-cognitive processes play in learning,
intelligent tutoring systems should not only provide domain-specific assistance, but
should also aim to help students in acquiring meta-cognitive skills. As a step toward this
goal, we have constructed a Help Tutor, aimed at improving students’ help-seeking
skill. The Help Tutor is based on a cognitive model of students’ desired help-seeking
processes, as they work with a Cognitive Tutor (Aleven et al., 2004). To provide metacognitive
tutoring in conjunction with cognitive tutoring, we designed an architecture in
which the Help Tutor and a Cognitive Tutor function as independent agents, to facilitate
re-use of the Help Tutor. Pilot tests with four students showed that students improved
their help-seeking behavior significantly while working with the Help Tutor. The
improvement could not be attributed to their becoming more familiar with the domainspecific
skills being taught by the tutor. Although students reported afterwards that they
welcomed feedback on their help-seeking behavior, they seemed less fond of it when
actually advised to act differently while working. We discuss our plans for an
experiment to evaluate the impact of the Help Tutor on students’ help-seeking behavior
and learning, including future learning, after their work with the Help Tutor
