126 research outputs found
MOOClets
No full textRandomized experiments in online educational environments are ubiquitous as a scientific method for investigating learning and motivation, but they rarely improve educational resources and produce practical benefits for learners. We suggest that tools for experimentally comparing resources are designed primarily through the lens of experiments as a scientific methodology, and therefore miss a tremendous opportunity for online experiments to serve as engines for dynamic improvement and personalization. We present the MOOClet requirements specification to guide the implementation of software tools for experiments to ensure that whenever alternative versions of a resource can be experimentally compared (by randomly assigning versions), the resource can also be dynamically improved (by changing which versions are presented), and personalized (by presenting different versions to different people). The MOOClet specification was used to implement DEXPER, a proof-of-concept web service backend that enables dynamic experimentation and personalization of resources embedded in frontend educational platforms. We describe three use cases of MOOClets for dynamic experimentation and personalization of motivational emails, explanations, and problems
The MOOClet Framework: Unifying Experimentation, Dynamic Improvement, and Personalization in Online Courses
No full textHow can educational platforms be instrumented to accelerate the use of research to improve students' experiences? We show how modular components of any educational interface-e.g. explanations, homework problems, even emails-can be implemented using the novel MOOClet software architecture. Researchers and instructors can use these augmented MOOClet components for: (1) Iterative Cycles of Randomized Experiments that test alternative versions of course content; (2) Data-Driven Improvement using adaptive experiments that rapidly use data to give better versions of content to future students, on the order of days rather than months. A MOOClet supports both manual and automated improvement using reinforcement learning; (3) Personalization by delivering alternative versions as a function of data about a student's characteristics or subgroup, using both expert-authored rules and data mining algorithms. We provide an open-source web service for implementing MOOClets (www.mooclet.org) that has been used with thousands of students. The MOOClet framework provides an ecosystem that transforms online course components into collaborative micro-laboratories, where instructors, experimental researchers, and data mining/machine learning researchers can engage in perpetual cycles of experimentation, improvement, and personalization. © 2021 ACM
Enhancing Online Problems Through Instructor-Centered Tools for Randomized Experiments
No full textDigital educational resources could enable the use of randomized experiments to answer pedagogical questions that instructors care about, taking academic research out of the laboratory and into the classroom. We take an instructorcentered approach to designing tools for experimentation that lower the barriers for instructors to conduct experiments. We explore this approach through DynamicProblem, a proof-ofconcept system for experimentation on components of digital problems, which provides interfaces for authoring of experiments on explanations, hints, feedback messages, and learning tips. To rapidly turn data from experiments into practical improvements, the system uses an interpretable machine learning algorithm to analyze students' ratings of which conditions are helpful, and present conditions to future students in proportion to the evidence they are higher rated. We evaluated the system by collaboratively deploying experiments in the courses of three mathematics instructors. They reported benefits in reflecting on their pedagogy, and having a new method for improving online problems for future students
Connecting Instructors and Learning Scientists via Collaborative Dynamic Experimentation
No full textThe shift to digital educational resources provides new opportunities to advance psychology and education research, in tandem with improving instruction using theory and data. To realize this potential, this paper explores how randomized experiments can support mutually beneficial instructor-researcher collaborations. We developed the Collaborative Dynamic Experimentation (CDE) framework to address two key tensions. To enable researchers to embed experiments in online lessons while maintaining instructors' editorial control, Collaborative experiment authoring is needed. To enable instructors to use data for rapid improvement while maintaining statistically valid data for researchers, we apply an interpretable machine learning algorithm for Dynamic experimentation. We worked with an on-campus instructor to implement a proof-of-concept CDE system to experiment within their online calculus quizzes. The qualitative results from this deployment provided insight into how the CDE framework can facilitate alignment of research and practice
Recommended from our members
The ABC's of Mathematics Perceptions
No full textWhat does the word "math" evoke? It is in many ways a fraught term eliciting negative reactions and unpleasant memories. In this dissertation, I explore the variety of ways we humans perceive this concept, starting with the internal (perceptions of ability), then the personal journey (perceptions of belonging), and finally the external (perceptions of conception). I employ a mixture of experiments and computational modeling in order to develop a more holistic understanding of how people perceive math and reinforce human studies with data collected from naturalistic settings - specifically, the internet, to explore word usage and how discussions of math seem to dier from mentions of other concepts
Automatic Subject-based Contextualisation of Programming Assignment Lists
No full textAs programming must be learned by doing, introductory programming course learners need to solve many problems, e.g., on systems such as ’Online Judges’. However, as such courses are often compulsory for non-Computer Science (nonCS) undergraduates, this may cause difficulties to learners that do not have the typical intrinsic motivation for programming as CS students do. In this sense, contextualised assignment lists, with programming problems related to the students’ major, could enhance engagement in the learning process. Thus, students would solve programming problems related to their academic context, improving their comprehension of the applicability and importance of programming. Nonetheless, preparing these contextually personalised programming assignments for classes for different courses is really laborious and would increase considerably the instructors’/monitors’ workload. Thus, this work aims, for the first time, to the best of our knowledge, to automatically classify the programming assignments in Online Judges based on students’ academic contexts by proposing a new context taxonomy, as well as a comprehensive pipeline evaluation methodology of cutting edge competitive Natural Language Processing (NLP). Our comprehensive methodology pipeline allows for comparing state of the art data augmentation, classifiers, beside NLP approaches. The context taxonomy created contains 23 subject matters related to the non-CS majors, representing thus a challenging multi-classification problem. We show how even on this problem, our comprehensive pipeline evaluation methodology allows us to achieve an accuracy of 95.2%, which makes it possible to automatically create contextually personalised program assignments for non-CS with a minimal error rate (4.8%)
Course Recommender Systems with Statistical Confidence
No full textSelecting courses in an open-curriculum education program is a difficult task for students and academic advisors. Course recommendation systems nowadays can be used to reduce the complexity of this task. To control the recommendation error, we argue that course recommendations need to be provided together with statistical confidence. The latter can be used for computing a statistically valid set of recommended courses that contains courses a student is likely to take with a probability of at least 1-e for a user-specified significance level e. For that purpose, we introduce a generic algorithm for course recommendation based on the conformal prediction framework. The algorithm is used for implementing two conformal course recommender systems. Through experimentation, we show that these systems accurately suggest courses to students while maintaining statistically valid sets of courses recommended
Course Recommender Systems with Statistical Confidence
No full textSelecting courses in an open-curriculum education program is a difficult task for students and academic advisors. Course recommendation systems nowadays can be used to reduce the complexity of this task. To control the recommendation error, we argue that course recommendations need to be provided together with statistical confidence. The latter can be used for computing a statistically valid set of recommended courses that contains courses a student is likely to take with a probability of at least 1-e for a user-specified significance level e. For that purpose, we introduce a generic algorithm for course recommendation based on the conformal prediction framework. The algorithm is used for implementing two conformal course recommender systems. Through experimentation, we show that these systems accurately suggest courses to students while maintaining statistically valid sets of courses recommended
- …
