68 research outputs found
Comparison and Parallel Implementation of Alternative Moving-Window Metrics of the Connectivity of Protected Areas Across Large Landscapes
No full textInputs, outputs and supplementary material for Hughes, J., Lucet, V., Barrett, G. et al. Comparison and parallel implementation of alternative moving-window metrics of the connectivity of protected areas across large landscapes. Landsc Ecol 38, 1411–1430 (2023). https://doi.org/10.1007/s10980-023-01619-9.
All code is available at https://github.com/LandSciTech/LSTD-Connectivity-Paper. R packages are available at https://github.com/LandSciTech/pfocal and https://github.com/LandSciTech/LSTDConnect
Comparison and Parallel Implementation of Alternative Moving-Window Metrics of the Connectivity of Protected Areas Across Large Landscapes
No full textInputs, outputs and supplementary material for Hughes, J., Lucet, V., Barrett, G. et al. Comparison and parallel implementation of alternative moving-window metrics of the connectivity of protected areas across large landscapes. Landsc Ecol 38, 1411–1430 (2023). https://doi.org/10.1007/s10980-023-01619-9.
All code is available at https://github.com/LandSciTech/LSTD-Connectivity-Paper. R packages are available at https://github.com/LandSciTech/pfocal and https://github.com/LandSciTech/LSTDConnect
Comparison and Parallel Implementation of Alternative Moving-Window Metrics of the Connectivity of Protected Areas Across Large Landscapes
No full textInputs, outputs and supplementary material for Hughes, J., Lucet, V., Barrett, G. et al. Comparison and parallel implementation of alternative moving-window metrics of the connectivity of protected areas across large landscapes. Landsc Ecol 38, 1411–1430 (2023). https://doi.org/10.1007/s10980-023-01619-9.
All code is available at https://github.com/LandSciTech/LSTD-Connectivity-Paper. R packages are available at https://github.com/LandSciTech/pfocal and https://github.com/LandSciTech/LSTDConnect
What Primary Schools Are Doing Right: Educational Value-Added in Luxembourg
No full textIn such a diverse context as Luxembourg, educational inequalities can arise from diverse languages spoken at home, a migration background, or a family’s socioeconomic status. This diversity leads to different preconditions for learning math and languages (e.g. the language of instruction) and thus shapes the school careers of students (Hadjar & Backes, 2021).
The aim of the project Systematic Identification of High Value-Added in Educational Contexts (SIVA) was to answer the questions (1) what highly effective schools are doing “right” or differently and (2) what other schools can learn from them in alleviating inequalities. In collaboration with the Observatoire National de la Qualité Scolaire, we investigated the differences of schools with stable high value-added (VA) scores to those with stable medium or low VA scores from multiple perspectives. VA is a statistical regression method usually used to fairly estimate schools’ effectiveness considering diverse student backgrounds.
First, we identified 16 schools which had a stable high, medium, or low VA scores over two years. Second, we collected data on their pedagogical strategies, student background, and school climate through questionnaires and classroom observations. Third, we matched our data to results from the Luxembourg School Monitoring Programme ÉpStan (LUCET, 2021). We selected the variables based on learning models focusing on aspects such as school organization or classroom management (e.g., Hattie, 2008; Helmke et al., 2008; Klieme et al., 2001). We further investigated specificities about the Luxembourgish school system, which are not represented in international school learning models (such as the division into two-year learning cycles, the multilingual school setting, or the diverse student population).
We will discuss the SIVA-project, its goals, and its data collection leading to data from observations in 49 classroom and questionnaires with over 500 second graders, their parents, their teachers, as well as school presidents and regional directors.
Literature
Hadjar, A., & Backes, S. (2021). Bildungsungleichheiten am Übergang in die Sekundarschule in Luxemburg. https://doi.org/10.48746/BB2021LU-DE-21A
Hattie, J. (2008). Visible Learning: A synthesis of over 800 meta-analyses relating to achievement (0 ed.). Routledge. https://doi.org/10.4324/9780203887332
Helmke, A., Rindermann, H., & Schrader, F.-W. (2008). Wirkfaktoren akademischer Leistungen in Schule und Hochschule [Determinants of academic achievement in school and university]. In M. Schneider & M. Hasselhorn (Eds.), Handbuch der pädagogischen Psychologie (Vol. 10, pp. 145–155). Hogrefe.
Klieme, E., Schümer, G., & Knoll, S. (2001). Mathematikunterricht in der Sekundarstufe I: “Aufgabenkultur” und Unterrichtsgestaltung. TIMSS - Impulse für Schule und Unterricht, 43–57.
LUCET. (2021). Épreuves Standardisées (ÉpStan). https://epstan.l
Stability of Value-Added Models: Comparing Classical and Machine Learning Approaches
No full textBackground: What is the value that teachers or schools add to the evolution of students’ performance? Value-added (VA) modeling aims to answer this question by quantifying the effect of pedagogical actions on students’ achievement, independent of students’ backgrounds (e.g., Braun, 2005). A plethora of VA models exist, and several outcome measures are in use to estimate VA scores, yet without consensus on the model specification (Everson, 2017; Levy et al., 2019). Furthermore, it is unclear whether the most frequently used VA models (i.e., multi-level, linear regression, and random forest models) and outcome measures (i.e., language and mathematics achievement) indicate a similar stability of VA scores over time.
Objectives: Drawing from the data of a highly diverse and multilingual school setting, where leveling out the influence of students’ backgrounds is of special interest, we aim to (a) clarify the stability of school VA scores over time; (b) shed light on the sensitivity toward different statistical models and outcome variables; and (c) evaluate the practical implications of (in)stable VA scores for individual schools.
Method: Utilizing the representative, longitudinal data from the Luxembourg School Monitoring Programme (LUCET, 2021), we examined the stability of school VA scores. We drew on two longitudinal data sets of students who participated in the standardized achievement tests in Grade 1 in 2014 or 2016 and then again in Grade 3 two years later (i.e., 2016 and 2018, respectively), with a total of 5875 students in 146 schools. School VA scores were calculated using classical approaches (i.e., linear regression and multilevel models) and one of the most commonly used machine learning approaches in educational research (i.e., random forests).
Results and Discussion: The overall stability over time across the VA models was moderate, with multilevel models showing greater stability than linear regression models and random forests. Stability differed across outcome measures and was higher for VA models with language achievement as an outcome variable as compared to those with mathematics achievement. Practical implications for schools and teachers will be discussed.Systematic Identification of High "Value-Added" in Educational Contexts - SIV
What Luxembourg's Primary Schools are Doing Right: A Value-Added Comparison in the Luxembourgish School Context
No full textKurz-Abstract (120 Wörter)
Luxemburgs Bildungssystem ist geprägt von multi-kulturellen und vielsprachigen Schüler:innen und einem zweimaligen Wechsel der Instruktionssprache. Dies führt zu sehr unterschiedlichen Voraussetzungen für die Schullaufbahn der Schüler:innen.
Das Ziel des vorliegenden SIVA-Projekts (Systematic Identification of High Value-Added in Educational Contexts) ist herauszufinden, welche pädagogischen Strategien Schulen mit hohen Value-Added (VA)-Werten für Schuleffektivität anwenden und was andere Schulen von ihnen lernen können, um diese Ungleichheiten abzubauen.
Zuerst ermittelten wir 16 Schulen, die stabil hohe, mittlere oder niedrige VA-Werte aufwiesen. Danach sammelten wir Daten anhand von Fragebögen und Unterrichtsbeobachtungen über pädagogische Strategien und das Schulklima und glichen sie mit repräsentativen Schulmonitoringergebnissen ab.
Wir werden das SIVA-Projekt, seine Ziele und die Datenerhebung diskutieren, die zu unserem reichhaltigen Datensatz aus sechs Perspektiven führte.
Zusammenfassung (480 Wörter)
In einem multi-kulturellen und vielsprachigen Land wie Luxemburg können leicht Bildungsungleichheiten entstehen. Unterschiedliche zu Hause gesprochene Sprachen, Migrationshintergründe oder der sozioökonomische Status einer Familie können zu ungleichen Erfolgschancen in der Schule werden. Gepaart mit einem Schulsystem, in dem zweimal die Instruktionssprache gewechselt wird, führt diese Vielfalt zu unterschiedlichen Voraussetzungen für das Erlernen von Mathematik und Sprachen und prägt somit die Schullaufbahn der Schüler:innen (Hadjar & Backes, 2021). Diese Gemengelage ist einerseits herausfordernd für Schüler:innen, Lehrkräfte und Schulen, zeigt aber andererseits, dass es gelingende soziale und pädagogische Praktiken geben muss, diese Herausforderungen zu meistern, da die Schulen weiterhin effektiv arbeiten.
In den USA wurde Schuleffektivität häufig mit Value-Added-Werten (VA) quantifiziert, welche durch ihre Instabilität zu ungerechtfertigten Finanzierungs- und Personalentscheidungen führten (Emslander, Levy, Scherer, et al., 2022). Ziel des Projekts Systematic Identification of High Value-Added in Educational Contexts (SIVA; Emslander, Levy, & Fischbach, 2022) ist es, dieses repressiv genutzte Instrument der VA-Werte konstruktiv anzuwenden. VA ist ein statistisches Regressionsverfahren, um die Effektivität von Schulen unter Berücksichtigung unterschiedlicher Schüler:innenhintergründe gerecht zu schätzen. Wir untersuchten, (1) was hocheffektive Schulen "richtig" machen und (2) was andere Schulen von ihnen lernen können, um Ungleichheiten abzubauen. In Zusammenarbeit mit der Section Qualité Scolaire des Observatoire National de l’Enfance, de la Jeunesse et de la Qualité Scolaire, untersuchten wir die Unterschiede zwischen Schulen mit stabil hohen, mittleren oder niedrigen VA-Werten aus verschiedenen Perspektiven.
Zunächst haben wir 16 Schulen ermittelt, die über zwei Jahre hinweg stabile hohe, mittlere oder niedrige VA-Werte aufwiesen. Als Zweites sammelten wir Fragebogen- und Unterrichtsbeobachtungsdaten über ihre pädagogischen Strategien, den Hintergrund der Schüler:innen und das Schulklima. Als Drittes glichen wir unsere Daten mit den Ergebnissen des luxemburgischen Schulmonitorings ÉpStan (LUCET, 2021) ab. Wir haben die Variablen auf der Grundlage von Lernmodellen ausgewählt, die sich auf Aspekte wie die Schulorganisation oder das Klassenmanagement konzentrieren (z.B. Hattie, 2008; Klieme et al., 2001). Darüber hinaus untersuchten wir die Besonderheiten des luxemburgischen Schulsystems, die in internationalen schulischen Lernmodellen nicht vertreten sind (z. B. die Einteilung in zweijährige Lernzyklen, die mehrsprachige Schulumgebung und die vielfältige Schülerschaft).
Wir werden das SIVA-Projekt, seine Ziele und Besonderheiten diskutieren, die zu Daten aus 49 Klassenzimmerbeobachtungen und Fragebögen mit über 500 Zweitklässler:innen, ihren Eltern, 200 Lehrkräften sowie Schulleiter:innen und Schulaufsichtsbehörden führte.
Literature
Emslander, V., Levy, J., & Fischbach, A. (2022). Systematic Identification of High “Value-Added” in Educational Contexts (SIVA). https://doi.org/10.17605/OSF.IO/X3C48
Emslander, V., Levy, J., Scherer, R., & Fischbach, A. (2022). Value-added scores show limited stability over time in primary school. PLOS ONE, 17(12), e0279255. https://doi.org/10.1371/journal.pone.0279255
Hadjar, A., & Backes, S. (2021). Bildungsungleichheiten am Übergang in die Sekundarschule in Luxemburg. https://doi.org/10.48746/BB2021LU-DE-21A
Hattie, J. (2008). Visible Learning: A synthesis of over 800 meta-analyses relating to achievement (0 ed.). Routledge. https://doi.org/10.4324/9780203887332
Klieme, E., Schümer, G., & Knoll, S. (2001). Mathematikunterricht in der Sekundarstufe I: “Aufgabenkultur” und Unterrichtsgestaltung. TIMSS - Impulse für Schule und Unterricht, 43–57.
LUCET. (2021). Épreuves Standardisées (ÉpStan). https://epstan.luSIV
Systematic Identification of High "Value-Added" in Educational Contexts (SIVA)
No full textThe aim of the SIVA project is to investigate differences between schools with stable high value-added (VA) scores to those with low or medium VA scores to learn about their effective pedagogical strategies. We attempt to achieve this goal through classroom observations and questionnaires for students in grade 2, their parents, their teachers, as well as school presidents. More specifically, with the present study we want to learn from target schools with stable positive VA scores – a statistical method usually used to estimate schools' effectiveness. We will use VA modelling constructively to compare those schools identified as highly effective (i.e., with high VA scores) to schools with medium or low VA scores on variables such as pedagogical strategies, student background, and school climate. To this end, a mixed-methods design based on questionnaires, observations, and results from the Luxembourg School Monitoring Programme ÉpStan (LUCET, 2021) will be applied. The content of the investigation is based on a synthesis of models of school learning and quality, focusing on aspects such as school organization or classroom management (e.g., Hattie, 2008; Helmke et al., 2008; Klieme et al., 2001) and is extended by specificities about the Luxembourgish school system, which are not represented in international school learning models (such as the division into two-year learning cycles, the multilingual school setting, and the diverse student population). With the aim to obtain a preferably broad picture, students, parents, teachers, school presidents and regional directors will be investigated. While parents, teachers, school presidents and regional directors can—as adults—fill out questionnaires individually, obtaining the opinion from children at such a young age can be challenging. The SIVA project tackles this issue by choosing item formats that are appealing and understandable for young children (see, e.g.,Lehnert, 2019), as well as by including classroom observations conducted by neutral educational experts (please, find both the questionnaires and observation sheets in the attachments).SIV
Are Value-Added Scores Stable Enough for High-Stakes Decisions?
No full textTheoretical Background: Can we quantify the effectiveness of a teacher or a school with a single number? Researchers in the field of value-added (VA) models may argue just that (e.g., Chetty et al., 2014; Kane et al., 2013). VA models are widely used for accountability purposes in education and quantify the value a teacher or a school adds to their students’ achievement. For this purpose, these models predict achievement over time and attempt to control for factors that cannot be influenced by schools or teachers (i.e., sociodemographic & sociocultural background). Following this logic, what is left must be due to teacher or school differences (see, e.g., Braun, 2005).
To utilize VA models for high-stakes decision-making (e.g., teachers’ tenure, the allocation of funding), these models would need to be highly stable over time. School-level stability over time, however, has hardly been researched at all and the resulting findings are mixed, with some studies indicating high stability of school VA scores over time (Ferrão, 2012; Thomas et al., 2007) and others reporting a lack of stability (e.g., Gorard et al., 2013; Perry, 2016). Furthermore, as there is no consensus on which variables to use as independent or dependent variables in VA models (Everson, 2017; Levy et al., 2019), the stability of VA could vary between different outcome measures (e.g., language or mathematics). If VA models lack stability over time and across outcome measures, their use as the primary information for high-stakes decision-making is in question, and the inferences drawn from them could be compromised.
Questions: With these uncertainties in mind, we examine the stability of school VA model scores over time and investigate the differences between language and mathematics achievement as outcome variables. Additionally, we demonstrate the real-life implications of (in)stable VA scores for single schools and point out an alternative, more constructive use of school VA models in educational research.
Method: To study the stability of VA scores on school level over time and across outcomes, we drew on a sample of 146 primary schools, using representative longitudinal data from the standardized achievement tests of the Luxembourg School Monitoring Programme (LUCET, 2021). These schools included a heterogeneous and multilingual sample of 7016 students. To determine the stability of VA scores in the subject of mathematics and in languages over time, we based our analysis on two longitudinal datasets (from 2015 to 2017 and from 2017 to 2019, respectively) and generated two VA scores per dataset, one for language and one for mathematics achievement. We further analyzed how many schools displayed stable VA scores in the respective outcomes over two years, and compared the rank correlations of VA scores between language and mathematics achievement as an outcome variable.
Results and Their Significance: Only 34-38 % of the schools showed stable VA scores from grade 1 to 3 with moderate rank correlations of r = .37 with language and r = .34 with mathematics achievement. We therefore discourage using VA models as the only information for high-stakes educational decisions. Nonetheless, we argue that VA models could be employed to find genuinely effective teaching or school practices—especially in heterogeneous student populations, such as Luxembourg, in which educational disparities are an important topic already in primary school (Hoffmann et al., 2018). Consequently, we contrast the school climate and instructional quality, which might be a driver of the differences between schools with stable high vs. low VA scores.
Literature
Braun, H. (2005). Using student progress to evaluate teachers: A primer on value-added models. Educational Testing Service.
Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014). Measuring the impacts of teachers I: Evaluating bias in teacher value-added estimates. American Economic Review, 104(9), 2593–2632. https://doi.org/10.1257/aer.104.9.2593
Everson, K. C. (2017). Value-added modeling and educational accountability: Are we answering the real questions? Review of Educational Research, 87(1), 35–70. https://doi.org/10.3102/0034654316637199
Ferrão, M. E. (2012). On the stability of value added indicators. Quality & Quantity, 46(2), 627–637. https://doi.org/10.1007/s11135-010-9417-6
Gorard, S., Hordosy, R., & Siddiqui, N. (2013). How unstable are “school effects” assessed by a value-added technique? International Education Studies, 6(1), 1–9. https://doi.org/10.5539/ies.v6n1p1
Kane, T. J., McCaffrey, D. F., Miller, T., & Staiger, D. O. (2013). Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment. Research Paper. MET Project. Bill & Melinda Gates Foundation. https://files.eric.ed.gov/fulltext/ED540959.pdf
Levy, J., Brunner, M., Keller, U., & Fischbach, A. (2019). Methodological issues in value-added modeling: An international review from 26 countries. Educational Assessment, Evaluation and Accountability, 31(3), 257–287. https://doi.org/10.1007/s11092-019-09303-w
LUCET. (2021). Épreuves Standardisées (ÉpStan). https://epstan.lu
Perry, T. (2016). English value-added measures: Examining the limitations of school performance measurement. British Educational Research Journal, 42(6), 1056–1080. https://doi.org/10.1002/berj.3247
Thomas, S., Peng, W. J., & Gray, J. (2007). Modelling patterns of improvement over time: Value added trends in English secondary school performance across ten cohorts. Oxford Review of Education, 33(3), 261–295. https://doi.org/10.1080/03054980701366116Systematic Identification of High "Value-Added" in Educational Contexts - SIV
Teacher-Student Relationships in Education—What We Know and What We Don’t (Yet) Know [SYMPOSIUM]
No full textPositive teacher-student relationships (TSR) are key to developing a good school climate in which both teachers and students can thrive. While existing research has brought to light the educational benefits of positive TSR, for instance, by showing that students in classrooms and schools with positive TSR tend to achieve better grades, the evidence base is scattered and lacks some key elements. Specifically, empirical studies on the benefits of positive TSR largely focused on academic achievement and less so on other, educationally relevant outcomes, such as socio-emotional skills, motivation, sense of belonging, or behavior. Moreover, TSR has often been conceptualized differently across studies, and its development in educational contexts has hardly been understood.
This symposium aims to clarify some of these issues by presenting studies that (a) review the conceptualizations and definitions of TSR within the frameworks of school climate; (b) synthesize the evidence base on the relation between TSR and a broad range of educationally relevant outcomes; (c) identify longitudinal trajectories of TSR and their relation to student engagement; and (d) examine the potential of TSR to facilitate a positive error culture and student participation in classrooms. Ultimately, we provide an updated, scientific overview of the existing body of knowledge about the conceptualization and educational potential of TSR and its current gaps. This overview shall not only inform scholars in the field but shall also encourage teachers to strive for positive TSR
Quality Assessment in Meta-Analyses (QuAMA)
No full textThis project develops methods and procedures to (a) quantify the quality of primary studies in meta-analyses; and (b) account for primary-study quality in moderator analyses. As part of the project, we develop an analytic procedure to create study quality indicators and incorporate them in the meta-analysis. We will present this procedure in a step-by-step tutorial with illustrative examples.QuAM
- …
