Journal of MultiDisciplinary Evaluation (JMDE)
Not a member yet
528 research outputs found
Sort by
Michael Scriven and “The Missing Half of Quantitative Evaluation”
Full text linkAlthough Michael Scriven is one of the most recognized parents of program evaluation in general, it may be surprising to learn that he also was a founder of cost-inclusive evaluation (CIE) in particular. CIE incorporates into evaluations information on the amounts, types, and monetary values of resources used by programs to achieve outcomes, and, increasingly, the monetary value of those outcomes (Herman et al., 2009; Persaud & Yates, 2023; Yates, 1996). Michael Scriven blazed a trail toward routine use of cost, cost-effectiveness, and cost-benefit analyses by emphasizing that evaluations should measure costs as well as nonmonetary and monetary outcomes, and should compare costs and outcomes for alternative programs in cost-effectiveness and cost-benefit analyses. Scriven also was emphatic that CIE be recognized as a core evaluator competency. Noting that “cost analysis” in evaluation was “the missing half of quantitative evaluation” (Scriven, 2008a, p. iii), he did not immediately succeed in convincing most evaluators that program costs, and monetary program outcomes, were essential to assess. However, his publications, his supervision, and his encouragement of junior colleagues legitimized and recognized CIE as nothing else could
The Evaluation Purposes Visualization
Full text linkBackground: The article outlines the development of a visualization that aimed to represent the purpose of different evaluation theories. This visualization used a stream/river metaphor to show the evolution of different purposes (e.g. utilization, knowledge generation, accountability) of evaluation, and also highlighted the historical context that had influenced their development. The visualization shows that the end goal of all these evaluation approaches (i.e. the ocean in which the rivers flow towards) is societal improvement.
Purpose: The article highlights the lessons learned and influences that led to the development of an evaluation theory categorization system. We hope that this would serve as an example (and maybe inspiration) for future efforts that aim to understand how evaluation theories have evolved and developed.
Setting: Not applicable.
Intervention: Not applicable.
Research Design: Not applicable.
Data Collection and Analysis: Not applicable.
Findings: Not applicable
A Map Of Uses Of Impact Evaluation Approaches
Full text linkBackground: Not all evaluation practitioners are experts in the diversity of impact evaluation approaches. How can these approaches be made clear and accessible? How can practitioners be better able to choose the relevant one? A collaborative project was initiated in 2021 to address these questions, and involved the French Development Agency’s evaluation officers, external evaluators, and public-sector co-designers.
Purpose: This article presents the development of a visual tool designed to facilitate meaningful discussions between evaluation officers and project managers regarding an array of available impact evaluation approaches. In our view, the positive reception of the map of uses of impact evaluation approaches within and beyond the Agence française de développement underscores the significance of anchoring discussions on evaluation scope and methodologies in real-world evaluation uses.
Setting: Not applicable.
Intervention: Not applicable.
Research design: Not applicable.
Data collection and analysis: Not applicable.
Findings: Not applicable
Even So, Too Soon: Michael John Scriven (28 March 1928 – 28 August 2023)
Full text linkA remembrance of the meetings between Glass and Scriven over more than fifty years
The Passing of a Giant and a Critical Friend: Michael Scriven
Full text linkA remembrance of Michael Scriven and his love and thrill of participating in great debates as a critical friend
Visualizing Evaluation Theory: Tree, Forest, or Ocean Currents?
Full text linkEvaluation theories depicted as a five-branch tree reflect the assumptions of the post-positivist, constructivist, pragmatic, transformative, and Indigenous paradigms. A tree is useful because it provides a quick and clear way to show that different assumptions lead to different methodological assumptions. However, evaluation theory as a five-branch tree is limited because it does not show interrelationships between the paradigms. An ocean current visualization could accomplish that task, but it might not be a familiar concept for some populations. Evaluators need to check with stakeholders to ensure the visualizations that are used are viewed as useful to their purposes
Behind That Intimidating Exterior: A Tribute to “Uncle Michael” Scriven
Full text linkFormer student and close colleague of the late Michael Scriven's for 26 years, Jane Davidson shares some insights into his personal side to shed light on where he was coming from as an evaluation theorist. 
The Importance of Conceptualizing Evaluation as a Transdiscipline: Honoring Michael Scriven’s Legacy
Full text linkStewart I. Donaldson's brief reflection on the importance of Michael Scriven's contributions to the conceptualization of evaluation as a transdiscipline
The Garden of Evaluation Approaches Visualization
Full text linkBackground: The Garden of Evaluation Approaches (Montrosse-Moorhead, et al., 2024a) maps evaluation approaches against eight dimensions of practice and situates them in their philosophical orientations and methodological dispositions.
Purpose: The Garden’s guiding question is: How do evaluation approaches compare in terms of dimensions that facilitate use and application? Primary intended users are evaluation practitioners, and secondary intended users include ecosystem actors (e.g., clients, program participants, funders), educators, researchers, and theoreticians.
Setting: Not applicable.
Intervention: Not applicable.
Research Design: Exploratory sequential mixed methods.
Data Collection and Analysis: Data collection and analysis included locating and reading primary sources for each evaluation approach. One team member thematically coded each source for evidence of an approach’s stance on the eight dimensions of practice, its’ philosophical orientation and methodological disposition, and steps for implementing the approach in practice. This qualitative evidence was used to generate ordinal ratings for the eight dimensions using a rating scale developed by the team, and to classify the approaches' philosophical orientation and methodological disposition. Drafts of handouts summarizing qualitative and quantitative data were reviewed by the two other team members, with dialogue and deliberation among all members used to come to consensus.
Findings: The first iteration of the Garden includes seven approaches: fourth generation evaluation, made in Africa, nation-to-nation evaluation, practical participatory evaluation, sistematización de experiencias, theory-driven evaluation, and transformative participatory evaluation. Each approach is represented as a flower within the Garden, with each petal corresponding to the approaches’ stance on one of the eight dimensions of practice. The colors of the flowers represent the underlying philosophical foundation of the approach. The pattern within the central disc of each flower represents the methodological disposition encouraged by the approach. Summaries of the empirical mixed-method analysis are made transparent in supporting handouts. Future work will grow flowers for additional approaches and continue to map out the Garden
The Periodic Table of Evaluation
Full text linkEvaluation is a vast field of designs and methods in continuous development. The numerous parameters and choices associated with different methodological options can be difficult to fully understand without deep study and desk review.
The motivation for the Periodic Table of Evaluation was to create a one-pager visual that catalogues methodological options and parameters involved in an evaluation design.