7 research outputs found
What Characterizes a Better Demonstration for Cognitive Modeling by Demonstration?
No full textA simulated student is a machine learning agent that learns a set of cognitive skills by observing solutions demonstrated
by human experts. The learned cognitive skills are converted into a cognitive model for a Cognitive Tutor that
is a computerized tutor that teaches human students the cognitive skills. In this paper, we analyze the characteristics
of the effective demonstrations that lead to quicker and more accurate learning. Results from empirical studies show
that expressive demonstrations (as opposed to abbreviated demonstrations that involve implicit mental operations)
are better for both speed and accuracy of learning. We also found that providing multiple demonstrations of the
same cognitive skill with differing surface features accelerates learning. These findings imply that the ordering of
training sequence as well as the level of detail in demonstration determines the efficiency with which a simulated
student generates a cognitive model
Applying Machine Learning to Cognitive Modeling for Cognitive Tutors
No full textThe aim of this study is to build an intelligent authoring environment for Cognitive Tutors in
which the author need not manually write a cognitive model. Writing a cognitive model usually requires
days of programming and testing even for a well-trained cognitive scientist. To achieve our goal, we have
built a machine learning agent – called a Simulated Student – that automatically generates a cognitive
model from sample solutions demonstrated by the human domain expert (i.e., the author). This paper
studies the effectiveness and generality of the Simulated Student. The major findings include (1) that the
order of training problems does not affect a quality of the cognitive model at the end of the training
session, (2) that ambiguities in the interpretation of demonstrations might hinder machine learning, and
(3) that more detailed demonstration can both avoid difficulties with ambiguity and prevent search
complexity from growing to impractical levels
Building Cognitive Tutors with Programming by Demonstration
No full textThe aim of this study is to incorporate the technique of programming
by demonstration (PBD) into an authoring tool for Cognitive Tutors. The primary
motivation of using PBD is to facilitate the authoring of Cognitive Tutors
by educators, rather than AI programmers. That is, instead of asking authors to
build a cognitive model representing a task to be taught, a machine-learning
agent – called the Simulated Student – observes the author performing the target
task and induces production rules that replicate the author’s performance. FOIL
is used to learn conditions appearing in the production rules. An evaluation in
an example domain of algebra equation solving shows that observing 10 problems
solved in 44 steps induced 9 correct and 1 wrong production rules. Two of
the correctly induced rules were overly general hence produced redundant solutions
Applying Programming by Demonstration in an Intelligent Authoring Tool for Cognitive Tutors
No full textWe are building an intelligent authoring tool for Cognitive
Tutors, a highly successful form of computer-based tutoring.
The primary target users (the authors) are educators who are
not familiar with cognitive task analysis and AI programming,
which are essential tasks in building Cognitive Tutors.
Instead of asking authors to write a cognitive model by hand,
a Simulated Student embedded in the authoring tool lets an
author demonstrate how to perform the tasks in the subject
domain, for instance, solving an algebra equation. The
Simulated Student observes an author’s demonstration and
induces a set of production rules that replicate the demonstrated
performances. Correct production rules, as well as
production rules that are incorrect but similar to those a
human student might produce, can be directly embedded in
the Cognitive Tutor. We give a preliminary evaluation of an
implemented Simulated Students based on inductive logic
programming and path-finding
Predicting Students’ Performance with SimStudent: Learning Cognitive Skills from Observation
No full textSimStudent is a machine-learning agent that learns cognitive skills by demonstration.
SimStudent was originally built as a building block for Cognitive Tutor Authoring Tools to help an author
build a cognitive model without significant programming. In this paper, we evaluate a second use of
SimStudent, viz., student modeling for Intelligent Tutoring Systems. The basic idea is to have SimStudent
observe human students solving problems. It then creates a cognitive model that can replicate the
students’ performance. If the model is accurate, it would predict the human students’ performance on
novel problems. An evaluation study showed that when trained on 15 problems, SimStudent accurately
predicted the human students’ correct behavior on the novel problems more than 80% of the time.
However, the current implementation of SimStudent does not accurately predict when the human students
make errors
Evaluating a Simulated Student using Real Students Data for Training and Testing
No full textSimStudent is a machine-learning agent that learns cognitive skills by demonstration. It was originally developed as a building block of the Cognitive Tutor Authoring Tools (CTAT), so that the authors do not have to build a cognitive model by hand, but instead simply demonstrate solutions for SimStudent to automatically generate a cognitive model. The SimStudent technology could then be used to model human students’ performance as well. To evaluate the applicability of SimStudent as a tool for modeling real students, we applied SimStudent to a genuine learning log gathered from classroom experiments with the Algebra I Cognitive Tutor. Such data can be seen as the human students’ “demonstrations” of how to solve problems. The results from an empirical study show that SimStudent can indeed model human students’ performance. After training on 20 problems solved by a group of human students, a cognitive model generated by SimStudent explained 82% of the problem-solving steps performed correctly by another group of human students
