Frontline Learning Research (E-Journal - EARLI, European Association for Research on Learning)
Not a member yet
256 research outputs found
Sort by
Disentangling objective characteristics of learning situations from subjective perceptions thereof, using an experience sampling method design
Full text linkThis article proposes a study design developed to disentangle the objective characteristics of a learning situation from individuals’ subjective perceptions of that situation. The term ‘objective characteristics’ refers to the agreement across students, whereas ‘subjective perceptions’ refers to inter-individual heterogeneity. We describe a novel strategy for assessing and disentangling objective situation characteristics and subjective perceptions thereof, propose methods for analyzing the resulting data, and illustrate the procedure with an example of a first study using this design to examine situational interest in 155 university students. Situational interest was assessed nine times per weekly lecture with three measurement time points per person and a rotated multi-group schedule. Assessments took place over the course of an entire semester of ten weeks.
One of the advantages of the proposed design is that ‘objective’ group agreements can be disentangled from subjective deviations from the group’s average at each of the nine measurement time points per weekly lecture. Furthermore, the proposed design makes it possible to study the development of both subjective and objective parameters across the time span of one weekly lecture and an entire semester, while the burden for each person is kept relatively low with three beeps per lecture
Ensuring content validity of psychological and educational tests – the role of experts
Full text linkMany test developers try to ensure the content validity of their tests by having external experts review the items in terms of relevance, difficulty, clarity, and so on. Although this approach is widely accepted, a closer look reveals there are several pitfalls that need to be avoided if experts’ advice is to be truly helpful. First, I offer a classification of tasks experts are given by test developers as reported on in the literature dealing with procedures of drawing on experts’ advice. Second, I review a sample of reports on test development (N = 72) to identify the common current procedures for selecting and consulting experts. Results indicate that often the choice of experts seems to be somewhat arbitrary, the questions posed to experts lack precision, and the methods used to evaluate experts' feedback are questionable. Given these findings I explore in more depth what prerequisites are necessary for their contributions to be useful in ensuring the content validity of tests. In conclusion, explicit guidelines on this matter need to be elaborated and standardised (above all, by the AERA, APA, and NCME-"Standards")
A Proposed Remedy for Grievances about Self-Report Methodologies
Full text linkThis special issue’s editors invited discussion of three broad questions. Slightly rephrased, they are: How well do self-report data represent theoretical constructs? How should analyses of data be conditioned by properties of self report data? In what ways do interpretations of self-report data shape interpretations of a study’s findings? To approach these issues, I first recap the kinds of self-report data gathered by researchers reporting in this special issue. With that background, I take up a fundamental question. What are self-report data? I foreshadow later critical analysis by listing facets I observe in operational definitions of self-report data: nature of the datum, topic, property, setting or context, response scale, and assumptions setting a stage for analyzing data. Discussion of these issues leads to a proposal that ameliorates some of them: Help respondents become better at self reporting
Variability in Certainty of Self-Reported Interest: Implications for Theory and Research: Implications for Theory and Research
No full textThese studies examined self-reported interest, how level of interest is related to reported certainty of interest, and whether certainty helps to clarify the relationship between interest and behavior. This research borrows from research on attitudes showing that attitude certainty helps to clarify the relationship between attitudes and behavior. A pilot study examined the relationship between self-reported interest and certainty of interest within four disciplines (math, psychology, biology, and astronomy). These relationships were replicated in math (Study 1) and psychology (Study 2), and the relationships between interest and behavior were stronger for those with greater certainty. For domains in which participants had sufficient levels of experience and varied levels of interest, curvilinear relationships were found between level of interest and certainty, showing that certainty is higher among individuals who report more extreme (high or low) levels of interest. Moreover, self-reported interest predicted behavior more strongly for those with more certainty in their responses. Discussion surrounds the theoretical and methodological utility of considering certainty of interest alongside measures of self-reported interest. 
A mixed method approach to studying self-regulated learning in MOOCs: combining trace data with interviews
Full text linkTo be successful in online education, learners should be able to self-regulate their learning due to the autonomy offered to them. Accurate measurement of learners’ self-regulated learning (SRL) in online education is necessary to determine which learners are in need of support and how to best offer support. Trace data is gathered automatically and unobtrusively during online education, and is therefore considered a valuable source to measure learners’ SRL. However, measuring SRL with trace data is challenging for two main reasons. First, without information on the how and why of learner behaviour it is difficult to interpret trace data correctly. Second, SRL activities outside of the online learning environment are not captured in trace data. To address these two challenges, we propose a mixed method approach with a sequential design. Such an approach is novel for the measurement of SRL. We present a pilot study in which we combined trace data with interview data to analyse learners’ SRL in online courses. In the interview, cued retrospective reporting was conducted by presenting learners with visualizations of their trace data. In the second part of the interview, learners’ activities outside of the online course environment were discussed. The results show that the mixed-method approach is indeed a promising approach to address the two described challenges. Suggestions for future research are provided, and include methodological considerations such as how to best visualize trace data for cued retrospective recall.  
Children’s Macro-Level Navigation Patterns in Hypermedia and their Relation with Task Structure and Learning Outcomes
Full text linkThis study investigated macro-level navigation patterns in children’s hypermedia learning, and how they related to task structure and learning outcomes. For this purpose, 5th and 6th grade learners performed a hypermedia assignment in which a high (n=57) versus a low (n=54) level of structure was provided. By means of qualitative analyses of their navigation activities, 6 macro-level navigation patterns were distinguished: linear reading, selective reading, video viewing, massed writing, late onset writing, and unpredictable reading. Results showed that the linear reading pattern was more frequent in the high structure environment, and that both the high structure environment and the linear reading pattern were associated with the highest quality of the children’s written assignments. Navigation patterns and task structure did not clearly predict children’s declarative knowledge gains or knowledge transfer. These findings show that there are multiple ways to navigate through a hypermedia environment, but that these are not all equally successful for learning. Moreover, the provided task structure in the environment may affect the occurrence of successful navigation patterns
The effect of first school years on mathematical skill profiles
Full text linkThis study investigated the effect of children’s first formal school years on mathematical skill profiles, measured by a variety of arithmetical skills and Spontaneous Focusing On Numerosity (SFON) tasks. By using person-centered approach the aim was to investigate whether the amount of formal schooling is associated with mathematical skills in the same way for all children, or, whether the associations differ according to the children’s mathematical skill profiles. Data was analyzed from 652 4–7-year-old children from four European countries with different school entrance ages. A person-centered approach with latent profile regression analyses on four-factor score variables identified six mathematical skill profiles with both qualitative and quantitative differences. The results revealed significant, but small effects of the amount of schooling on mathematical profiles when chronological age and country-specific school entrance age were controlled for. Educational implications of the findings emphasize regarding the heterogeneity in children’s mathematical skill profiles and the potentially different effects of starting formal schooling across different profiles
Peer selection and influence: Students’ interest-driven socio-digital participation and friendship networks
Full text linkDigital technologies have been increasingly embedded in students’ everyday lives. Interest-driven socio-digital participation (ISDP) involves students’ pursuit of interests mediated by computers, social media, the internet, and mobile devices’ integrated systems. ISDP is likely to intertwine closely with young people’s social networks that has been scarcely studied quantitatively. To close this gap, the present paper investigated students’ peer selection and influence effects of the intensity of their ISDP and friendship networks. We collected two-wave data by administering a peer nomination to trace students’ friendship networks with peers and a self-reported questionnaire to examine students’ ISDP. Participants were 100 students in Finland (female: 53%; mean age = 13.48, in grade 7 in the first wave). Through stochastic actor-oriented modelling, the results showed that the students’ friendship ties with peers influenced the intensity of their ISDP practices to become more similar. Yet, students did not select peers as friends based on similar intensity levels of ISDP. Utilizing influence effect found in students’ ISDP and their peer networks, we suggest that connected learning (Ito et al., 2013) should be promoted to integrate students’ informal and formal learning in order to bridge the gap between students’ informal interest-related digital practices and formal educational practices
Assessing the Predictive Nature of Teacher and Student Writing Self-Regulation Discrepancy
Full text linkIn this study, we examine the extent to which the discrepancy between teacher-reported and student-reported self-regulatory behaviours during writing were associated with students’ end-of-year writing grades after controlling for student writing ability and other demographic characteristics. Results of our study, conducted with a sample of 201 middle grades students enrolled in a large, comprehensive suburban school district in the Mid-Atlantic U.S., suggest a significant and positive relationship between teacher discrepancy and grades, after controlling for writing ability, student self-regulation, gender, race/ethnicity, and SES. This has clear implications for the classroom, as it suggests that even after accounting for student difference in terms of ability background, and demographics, the effort that teachers perceive their students making in the fall are still associated with students’ year-end performance in their class. This represents some of the first frontline evidence of the predictive relationship between self-regulation discrepancy and student achievement in writing
Tracking Patterns in Self-Regulated Learning Using Students’ Self-Reports and Online Trace Data
Full text linkFor decades, self-report instruments – which rely heavily on students’ perceptions and beliefs – have been the dominant way of measuring motivation and strategy use. An event-based measure based on online trace data arguably has the potential to remove analytical restrictions of self-report measures. The purpose of this study is therefore to triangulate constructs suggested in theory and measured using self-reported data with revealed online traces of learning behaviour. The results show that online trace data of learning behaviour are complementary to self-reports, as they explained a unique proportion of variance in student academic performance and reveal that self-reports explain more variance in online learning behaviour of prior weeks than variance in learning behaviour in succeeding weeks. Student motivation is, however, to a lesser extent captured with online trace data, likely because of its covert nature. In that respect, it is of importance to recognize the crucial role of self-reports in capturing student learning holistically. This manuscript is ‘frontline’ in the sense that event-based measurement methodologies using online trace data are relatively unexplored. The comparison with self-report data made in this manuscript sheds new light on the added value of innovative and traditional methods of measuring motivation and strategy use