Journal of MultiDisciplinary Evaluation (JMDE)
Not a member yet
528 research outputs found
Sort by
Evaluation as the Conscience of Freedom and Democracy
Get PDFIt wasn’t long ago that democratic political systems along with the free-market forces of globalization were generally expected to spread unchecked throughout the world. Only five years ago most would have found it unbelievable that the leading democratic free-market economies of the world would have significant trouble competing with non-democratic state-run economies and far flung terrorist organizations. Now this is obviously the case, and these competitors are aggressively spreading their influence around the world. This paper suggests the perhaps counter-intuitive notion that improvements to the theory and practice of evaluation can play a perhaps critical role in improving the relative performance of market-based democratic systems and explores what some of these potential improvements might entail
Addendum to "A Review of the History and the Current Practice of Aid Evaluation"
Get PDFIn the article in the previous issue of JMDE, September 2006, “A Review of the History and the Current Practice of Aid Evaluation,” I stated “the EU does not have a unified evaluation approach, because a different EU member agency takes the lead in each evaluation project.” At the time when I accessed the webpage of Europe Aid Co-operation Office (AIDCO), the guidelines of evaluation were not available. The Webpage simply stated “the methodology pages will be available in English, French and Spanish and will be accessible early 2006.” It can be found at: http://ec.europa.eu/europeaid/evaluation/methodology)
Evaluation as a Cognitive Process
Get PDFEvaluation is one of a number of so-called ‘higher order’ cognitive processes that are involved in the brain’s survival activities, including those extensions of basic coping processes (such as concept formation) that go into scientific and technological developments (like hypothesis testing). Some of these have been brought into various theories of learning and teaching. For example, in the (Benjamin) Bloom taxonomy, evaluation is listed as the intellectual activity at the top of the pyramid of skills at which teaching can be aimed. In this note, we’ll look at some of the other cognitive processes that need to be distinguished from evaluation, and look at unpacking evaluation into some component processes. Doing this will help us avoid some confusions that hamper effective evaluation work, and also provide another kind of foundation for an epistemology and logic of evaluation
Review of Evaluation and Program Planning
Get PDFEvaluation and Program Planning is a quarterly publication aimed at a broad audience of professional evaluators, individuals involved in program planning and management, and academic social scientists. Its contributors come from a wide range of backgrounds, fields, and viewpoints such as government, public health, education, policy, professional evaluation, and the non-profit sector. One consistent feature, a requirement that submissions address a practical “lesson learned”, helps make this journal especially useful in the promotion of hands-on evaluation practice, since even articles with a narrow focus must contain material that is relatable to the broader world of evaluation. Additionally, many issues also devote a substantial portion of the journal to a specific topic, making a wide range of subject-matter perspective easily available to the reader. The latest issue of Evaluation and Program Planning (volume 29 issue 2, 2006) contains a special topic section on “Program Capacity and Sustainability” that will be of particular interest to those involved/with non-profit sector organizations
Using Multivariate Techniques to Measure the Performance of R&D Programs: A Case Example
Get PDFPerformance management systems implemented in science-based government organizations have traditionally focused on research inputs and activities, rather than outputs or outcomes. However, recent legislative changes in several countries now require individual programs to report on their progress towards the achievement of organizational and governmental strategic objectives. In a substantive field where peer review remains the standard evaluation method against which scientific success is judged, performance measurement activities have often been articulated around complex techniques taken from the sciences and economics that yield little useful information to key decision makers (Geisler, 2002; McDonald & Teather, 2000; Roessner, 2002)
Asymmetrical Tests
Get PDFIt is often thought, or presupposed, that investigative tests done for evaluative purposes should be symmetrical—that is, they should be equally capable of giving positive and negative results. This is an error in the logic of evaluation, and it arises from confusion between the asymmetrical ability of a test to yield information and its propensity to yield biased information. Although ‘asymmetrical’ does connote one-sidedness and we often use ‘one-sided’ to mean biased, we can and should use the terms more carefully. Some tests can only yield information about the faults of a program (or product, or person, etc.)—or only about its virtues—while others can yield one of these more reliably than the other, and yet others are symmetrical in their treatment of merits and demerits. All of them gather relevant information for evaluative purposes, and one cannot conclude that any of them are biased simply because they are asymmetrical. The point is important because in many situations, one may only have access to asymmetrical tests and this does not support the claim of a biased approach as long as there is more than one test in the battery used to evaluate, and one test’s asymmetry is balanced out by the other test(s)
Review of the Japanese Journal of Evaluation Study
Get PDFThe Japan Evaluation Society was established in 2000. Since then, its professional journal, called Japanese Journal of Evaluation Study (JJES), has been published twice a year. These issues can be downloaded from the Society’s website (http://www.idcj.or.jp/JES). We can clearly observe several hot issues recently discussed by Japanese evaluation professionals in the journal. In this paper, three of them discussed in the 2005 issues are explained briefly
Formative Evaluation of an Educational Technology Innovation: Developer's Insights into Assessment Tools for Teaching and Learning
No full textFormative evaluations conducted by the development team, focusing on the consequences of choices to be made by the developers, during the development of asTTle, a new ICT2-mediated educational assessment resource, are reported. The evaluations focused on the validity, accuracy, added value, training, and utility standards identified as important in the deployment of educational technology. As a result of in-house utilization of the evaluations, it is argued that users can have confidence that asTTle has demonstrated validity through alignment with the curriculum and classroom practice, that the reporting systems add value to teachers’ work and that accuracy of understanding is enhanced by professional development. Insights into effective PD have been obtained, though further improvements in the training component of the system are required. The utility of the software has been constantly increased, but there are still unresolved issues that are currently being responded to. The gradual implementation mechanism has been successfully used to obtain user buy-in as well as identification and response to identified needs. Users and funding agencies can have confidence that asTTle has been designed, developed, and implemented in a fashion consistent with maximizing benefits to the end-user. The software has been developed in such a way that the validity, added value, accuracy, training and utility standards for educational technology have been met, at least in part. The data reported here demonstrate again that formative evaluations which maximize end-user benefits can add significant value to an educational technology innovation
The Fundamental Characteristics of Research
Get PDFOne of the most serious problems encountered in evaluating research, or researchers, is the dispute about what is and is not research. That is, how is research defined? Or, can it be satisfactorily defined? Of course, there are numerous competing definitions of the term and little apparent consensus. However, common usage holds that research is a truth-seeking activity which contributes to knowledge, aimed at describing or explaining the world, conducted and governed by those with a high level of proficiency or expertise.