Journal of MultiDisciplinary Evaluation (JMDE)
Not a member yet
528 research outputs found
Sort by
Handbook of Constructionist Research
Get PDFA review of the book Handbook of Constructionist Research edited by James A. Holstein and Jaber F. Gubrium, published in 2008 by Guilford Press
Contexts and Constraints: An Analysis of the Evolution of Evaluation in Ireland with Particular Reference to the Education System
Get PDFThis paper is a case study of the emergence of an evaluation culture in the public sector and particularly in education in Ireland over the past three decades. It suggests that the emergence of this culture was strongly influenced by external factors, particularly the European Union (EU), and to a lesser but significant degree, the Organisation for Economic Cooperation and Development (OECD). Indeed, it can be argued that without these external influences no culture of evaluation would have emerged. Moreover it is further suggested that even after three decades the continuation of systematic evaluation is still probably dependent on external forces, since a belief in or commitment to evaluation as a tool of governance has not taken hold among key policy makers in Ireland. A second point made in the paper is that, while evaluation practice in Ireland was at first largely concentrated around EU funded projects and economic transfers, in more recent times evaluation has migrated to programmes and policies. In particular, it has become an element of the quality assurance processes institutionalised as part of the ‘reform agenda’ of the public service. Finally, the paper makes the point that an evaluation culture in a particular country—in this case, Ireland, but the same applies elsewhere—is hugely contextualised and influenced by the constraints of existing ideologies, traditions, practices and relationships between different interest groups. Thus, in Ireland, in line with the corporatist and partnership-driven approaches to economic policy and industrial relations which have been dominant in the past two decades, the form of evaluation which has emerged is consensual, collaborative and negotiated
Culturally Competent Evaluation for Aboriginal Communities: A Review of the Empirical Literature
Get PDFMuch of program evaluation is concerned with understanding and improving social programs so that they are ultimately more responsive and more reflective of program participant needs. At the same time, these programs exist and are embedded within specific social, cultural and historical contexts which impact program development, implementation, and eventual outcomes. Evaluations that attempt to address responsiveness to contextual and cultural specificity are often referred to as culturally competent, culturally responsive, inclusive, multicultural, or cross-cultural, among other terms. While there are no agreed upon terminologies, definitions, or even methodologies, what these approaches all share is the recognition that culture and context matter, and that there are no universally agreed upon rules or abstractions that can be applicable in all contexts (Guba & Lincoln, 2005). The recognition of culture and context thus becomes “an explicit criterion rather than an unspoken expectation” (SenGupta, Hopson, & Thompson-Robinson, 2004, p. 15) in evaluations of this type
Predictive Evaluation
Get PDFThere has been extensive discussion of the relation of evaluation to: (i) research; (ii) explanations (a.k.a. theory-driven, logic model, or realistic evaluation); and (iii) recommendations, from each of which it is logically distinct, although there are times when they do and should overlap in practice. Making the distinction is not a mere linguistic issue since it vitally affects practice and the whole process of learning and teaching evaluation, and responsibly providing evaluation services. It is now time to look more carefully at the ghost at the banquet, (iv) prediction
Are National-Level Research Evaluation Models Valid, Credible, Useful, Cost-Effective, and Ethical?
Get PDFThe evaluation of government-financed research has become increasingly important in the last few decades in terms of increasing the quality of, and payoff from, the research that is done, reducing the cost of doing it, and lending public credibility to the manner in which research is funded. But there are very large differences throughout the world in the extent to which systems used promote these results. This paper briefly presents the dimensional results of a study designed to comparatively evaluate the national-level research evaluation models in sixteen countries on five merit-defining dimensions
Programme Evaluation Theory: The Next Step Toward a Synthesis of Logic Models and Organizational Theory
Get PDFThis paper argues that the formulation of policy, at whatever level, to whatever scale or scope, is any different to the myriad of processes involved in strategic planning within and between organisations, and the attendant decision making processes that abound in such an environment (Hage, 1980; Hickson, 1987; Thompson, 1967; Weick, 1976). Those forces that impact upon organisations are precisely the same forces that impact upon policy making groups who are themselves ‘organisations’ (or institutions), whether actual or representative. This provides a startign point for considering a synthesis between various models or paradigms. The argument continues, that program theory and the program logic model can incorporate and synthesise theories from elsewhere that, in turn, usefully inform and develop programme theory and program logic. A ‘hierarchical’ model is presented to explore the linkages between the components and ‘simple’ processes of theory, programme theory, logic modelling, organisational functioning, policy, and the consequent impact. It is hoped that this model can be used as a framework to: (a) determine which components are essential to give the policy external validity, credibility, implementability, and so on; and (b) provide a common point of reference for policy makers, stakeholders, and evaluators
The 'Holy Trinity' of Methodological Rigor: A Skeptical View
Get PDFRigor in research is normally conceived of as the means by which integrity and competence are confirmed (Tobin & Begley, 2004).1 That is, a way of demonstrating the legitimacy or soundness of the research process. Without rigor, it is argued, there is a danger that research may become fictional journalism and therefore worthless as contributing to knowledge (Morse, Barrett, Mayan, Olson, & Spiers, 2002)
On Yesbuts and Whynots
Get PDFA poem reflecting on "Yesbuts" and "Whynots" in the evaluation practice.
Evaluation in the Context of the Government Market Place: Implications for the Evaluation of Research
Get PDFThe evaluation community has concentrated on examining and explicating implications of the choice of methods for evaluating federal programs, as described in the New Directions for Evaluation edited by Julnes and Rog (2007), placing the policy debate in historical and contemporary contexts. In that volume and elsewhere we find that there are several mechanisms described for supporting and/or conducting program evaluation at the federal level. In the Julnes and Rog volume, Chelimsky (2007) describes evaluation activities conducted within the federal government by the Government Accountability Office (GAO). Both grants and contracts supported the work of Yin (Yin and Davis, 2007) in the evaluation of large comprehensive reforms in K-12 science and mathematics education. Other evaluation activities come under the authority of the Office of the Inspector General which conducts performance audits of government programs that draw on program evaluation and its methods (see the Yellow Book http://www.gao.gov/govaud/ybk01.htm)
The Logic of Summative Confidence
Get PDFThe constraints of conducting evaluations in real-world settings often necessitate the implementation of less than ideal designs. Unfortunately, the standard method for estimating the precision of a result (i.e., confidence intervals [CI]) cannot be used for evaluative conclusions that are derived from multiple indicators, measures, and data sources, for example. Moreover, CIs ignore the impact of sampling and measurement error. Considering that the vast majority of evaluative conclusions are based on numerous criteria of merit that often are poorly measured, a significant gap exists with respect to how one can estimate the CI of an evaluative conclusion. The purpose of this paper is (1) to heighten reader consciousness about the consequences of utilizing a weak evaluation design and (2) to introduce the need for the development a methodology that can be used to characterize the precision of an evaluative conclusion