Journal of MultiDisciplinary Evaluation (JMDE)
Not a member yet
528 research outputs found
Sort by
Examining Teacher Candidate Use of Data-Based Formative Assessment for Instructional Decision-Making
Get PDFUsing accounts of critical events, this study examines teacher candidate decisions to modify instruction in response to ongoing classroom interaction and assessment during student teaching. Instructional changes made by candidates who reported incidents in which students had difficulty with learning goals were explored across two years. Analysis of decision making suggested that candidates during the first year lacked variation in instructional interventions and continued presenting content to students through previously used teaching strategies and materials. In the second year, candidates improved their instructional interventions, but had difficulty using objective data to describe outcomes resulting from changes made. The discussion examines changes made to teacher education curriculum after study year one to prompt teacher candidate recognition of and accommodation for diverse student needs when making instruction and assessment decisions. The use of data in reflective practice is argued as a continued need for teacher candidate development
Unintended Consequences of using Tests to Improve Learning: How Improvement-Oriented Resources Heighten Conceptions of Assessment as School Accountability
Get PDFOver the past decade, the New Zealand government has created a set of resources to support teachers’ use of assessment for learning. These include Assessment Tools for Teaching and Learning (asTTle), a software program enabling teachers to create personalized but standardized tests for diagnostic purposes, and Assess to Learn (AtoL), an intensive professional development program. These resources were expected to increase teacher agreement that improvement is the major purpose of assessment. Instead, a 2008 sample of teachers completing the Teacher Conceptions of Assessment questionnaire showed significantly higher agreement that assessment is about school accountability than participants in previous national surveys. Unlike previous surveys, the correlation between school accountability and improvement conceptions was not statistically significant. However, as only the improvement conception predicted the practices teachers used to define assessment (β = .32), it appears that these teachers still saw many of the practices they used in the classroom (e.g. oral and interactive assessments) as improvement-oriented. Interviews with twenty-six of the surveyed teachers identified that while a few saw the new resources as contributing to improvement and accountability purposes, a larger group failed to make that connection. This second group seemed to be unable to accept that tests, an assessment genre traditionally associated with school and student accountability, could be meaningfully used for improvement at the classroom level. These data show that schools and individuals mediate the implementation of any policy initiative and can therefore cause it to have a range of often unintended consequences. In light of this, the thinking of teachers and other educational stakeholders should be taken into account when enacting policy changes
Using Assessment and Feedback to Enhance Learning: Examining the Relationship Between Teachers’ Reported Use of Assessment and Feedback and Student Performance in AP Biology
Get PDFThis paper analyzes a national sample of teachers’ self-reported use of assessment and feedback in Advanced Placement® (AP) biology classrooms. Descriptive statistics of what teachers reported doing in these two areas are discussed, followed by the identification of reported teaching practices variables that were found to be significantly related with student performance on AP biology exams. All the significant variables found are exemplars of authentic assessment and feedback practices and techniques. The study provides valuable information on the types of assessment and feedback practices currently being employed by teachers, and shows the relationship of such practices with actual student outcomes
In This Issue
Get PDFThis issue of JMDE centers around a grant-funded project currently being conducted by The Evaluation Center, the Mallinson Institute for Science Education, and Mathematics Education at Western Michigan University. This project focuses on Assessment for Learning (AfL). Our editorial in this issue is brought to us by Paul Black, Emeritus Professor of Science Education at King’s College in London, England
Comments and Proposals Concerning Chianca’s “The OECD/DAC Criteria for International Development Evaluations: An Assessment and Ideas for Improvement”
Get PDFHaving been involved in the discussions preceding the establishment of the OECD/DAC criteria as member of the DAC Working Party on International Development Cooperation Evaluation (IDCE) representing the European Commission, it is understandable that I have read Chianca’s (2008) article with exceptional interest. There is hardly anything in this article that I would disagree with, except a few minor aspects that I will mention in due course. I also fully support the idea of taking a critical look at the DAC criteria and at IDCE approaches in general, a critical review that should always continue as an ongoing process. “If followed,” Chianca says the changes he proposes “could contribute for increasing the quality of evaluation for the purposes of . . . decision options” (p. 1). I will try to formulate some proposals with a view to bringing that “If followed” premise a bit closer to reality. My comments have been written with the contents of my article, “Planning and Evaluation, Two Sides of the Same Coin,” in mind and should be understood in that context. (Eggers, 2006).
Meta-Evaluation Revisited
Get PDFNext month will mark the passage of forty years since I published my first article about “meta-evaluation” (Scriven, 1969), a term I had invented somewhat earlier in a report to the Urban Institute, who had asked me for help in dealing with the noncomparability of the evaluations they had commissioned for several housing projects. This year also marks the elevation of the concept to the status of an independent category in the latest edition of the Program Evaluation Standards (Joint Committee, 1994). So the editors of JMDE thought it might be time to take another look at this process and its products, and, as promised, we have provided a special section in this issue on the topic. Many thanks for the much-appreciated contributions that were sent in for that section! Before reading them, and hence without implying any criticism of them for ignoring or disagreeing with what follows, I’m going to mention some reflections on the notion that now seem to me worth stressing. They—and the other articles in this issue—may inspire reactions from you: Please put them on paper or screen, and we’d be happy to consider them for the (nonrefereed) discussion section of a later issue of JMDE
The Effects of Professional Development in Formative Assessment on Mathematics Teaching Performance and Student Achievement
Get PDFThis article describes a year-long professional development program on formative assessment for middle school mathematics teachers and its impact on teachers’ use of formative assessment during instruction. The focus of the program was to support teachers in (a) embedding formative assessments into their instruction, (b) interpreting student responses and performance on formative assessments, (c) using effective questioning strategies and peer assessment, and (d) identifying student errors and misconceptions and implementing instructional strategies to address them. Teacher and student performance data were collected on participating teachers, control teachers, and their students during the year-long professional development. The professional development had an impact on teachers’ cognitive level of questioning, use of peer assessment, and types of questioning strategies. Results of student performance data revealed differences in performance on some types of items among students of participating teachers and control teachers
Curriculum Evaluation in Greece
Get PDFThe role of curriculum in the official educational process is widely recognized by the international scientific community. Beginning in the 19th century, and perhaps even earlier, curriculum research began, not only to be systematized, but to also constitute an autonomous field of study. On the other hand, curriculum evaluation has captured the attention of experts involved in the science of education over the last few years. This article will present the results arising from research with respects to curriculum evaluation in Greece along with a micro-model proposed for the evaluation of the Cross-Thematic Unified Curriculum Framework and the corresponding Individual Subject Curricula that are applied today in compulsory Greek education
Nonmarket Economic Valuation and "The Economist’s Fallacy"
Get PDFIt is important that noneconomists understand and consider three key points regarding the use of economic cost-benefit analysis. First, economists undertaking applied cost-benefit analyses use expected values by necessity in forward-looking models. Second, economists tally changes in economic value, comparing “with” and “without” investment scenarios, using producer and consumer surplus. Third, marginal changes in the “nonmarket” components of consumer surplus can, and should, be quantified in monetary terms and included in evaluations whenever possible. These three factors highlight the need for multidisciplinary research on complex problems relating public investments to the social, cultural, health, and environmental impacts of those investments
Investigating Teacher Use of Practice Tests for Formative Purposes
Get PDFThis paper describes an intensive professional development program focused on supporting teachers at high-need urban high schools in use of practice tests for formative assessment purposes. Teachers engaged in professional development while offering intensive summer school to students with histories of failure on state tests in science and mathematics. Specific skills that teachers developed included item analysis of practice tests, planning around assessment results, and providing feedback to students. The study used a quasi-experimental pretest/posttest design with an intervention and a comparison group, analysis of classroom artifacts, follow-up surveys, and long- term follow-up with teachers. Students in the summer programs experienced large gains in performance, and teachers reported more frequent and varied use of tests for formative purposes, even one-and-one-half years after the end of training.
This material is based upon work supported by the National Science Foundation’s Math and Science Partnership program under Grant No. 041241