Journal of MultiDisciplinary Evaluation (JMDE)
Not a member yet
528 research outputs found
Sort by
An Interview with Ralph Tyler
No full textOriginally published as Paper #13, Occasional Paper Series, November, 198
Standards and Criteria*
Get PDFOriginally published as Paper #10, Occasional Paper Series, December, 1977
* This monograph was prepared under a grant from the Carnegie Corporation. Mary Lee Smith contributed at every stage of the preparation of this monograph; reading the literature, developing the ideas, outlining and editing the final product. Nancy W. Burton helped in developing the topic. Laura A. Driscoll and Gregory C. Camilli deserve thanks for assistance in literature search and data analysis. Conversation among Glass, Smith, Burton, and Robert Glaser on a day in June 1976 got things started; we are much indebted to Glaser for helping set things off in a direction that now pleases us
A Possible New Role in the Evaluation Process
Get PDFThe usual players in the ‘evaluation game’ might be listed as below. I think this list is a bit more extensive than any I've seen before but if you think of something I’ve missed, please tell me ([email protected]) or write a response for our comments department at jmde.com. To this cast I’m going to add one more role, which I'll call the Evaluation Advisor, defined below. People have served in this role, but I do not know of any discussion of it, especially the particular version of it described here
Content Specialization and Educational Evaluation: A Necessary Marriage
No full textOriginally published as Paper #14, Occasional Paper Series, September, 198
The Pragmatist in Context of a National Science Foundation Supported Grant Program Evaluation: Guidelines and Paradigms
Get PDFBackground: The philosophical underpinnings of evaluation guidelines set forth by a funding agency can sometimes seem inconsistent with that of the intervention.
Purpose: Our purpose is to introduce questions pertaining to the contrast between the instructional program’s underlying philosophical beliefs and assumptions and those underlying our evaluation approach. Drawing heavily on Scriven, we discuss these from a pragmatist evaluation stance in light of issues defined by Lincoln and Guba (2000). The discussion is couched in the evaluation of an innovative approach to teaching computer science.
Setting: Auburn University, Auburn, AL
Intervention: The evaluation is designed to investigate the effects of a studio-based teaching approach in computer science education. The evaluation framework employs a rigorous design that seeks to provide evidence to support or refute some assumed truth about the object (or construct) investigated. The program evaluated is steeped in a constructivist framework which assumes that no universal truth or reality exists, but rather, is constructed by the individual.
Research Design: Our evaluation design, to a good extent, reflects a post-positivist, quasi-experimental position. We also include a qualitative component using student interviews.
Data Collection and Analysis: Evidence of the effectiveness of the instructional approach for learning is assessed quantitatively using pre- and post-test and pre- and post-survey data group comparisons (mixed design ANOVA). Interviews provide the basis for qualitative theme analysis.
Findings: Quantitative results were somewhat weak but consistent in support of the studio-based teaching. Interview data suggest that most students did find working in groups enjoyable and a valuable experience
Evaluation Bias and its Control*
No full textOriginally published as Paper #4, Occasional Paper Series, June, 1975
*A paper originally written in conjunction with The Evaluation Center’s Grant # GW-7903 from the National Science Foundation
Needed: Instruments as Good as Our Eyes*
No full textOriginally published as Paper #7, Occasional Paper Series, July, 1976* This paper was originally published in the Journal of Career Education, Winter 1976, 2(3), 56-66
A Basis for Determining the Adequacy of Evaluation Designs
No full textOriginally published as Paper #7, Occasional Paper Series, April, 1976
* This paper was prepared under contract support from the Alaska Department of Education to the Northwest Regional Educational Laboratory
Demands on Users for Interpretation of Achievement Test Scores: Implications for the Evaluation Profession
Get PDFBackground: Professional standards for validity of achievement tests have long reflected a consensus that validity is the degree to which evidence and theory support interpretations of test scores entailed by the intended uses of tests. Yet there are convincing lines of evidence that the standards are not adequately followed in practice, that standards alone are not sufficient guides to action, and that reviewers of tests do not call attention to important kinds of validity evidence that might support the demanding process of making sense of test scores or reasoning from test scores.
Purpose: The intent of this article is to make more transparent the demands of achievement test interpretation on users in instructional contexts and to open up a dialogue on implications for the evaluation profession for improvement of practice along lines already set out by evaluation theorists.
Setting: Not applicable.
Intervention: Not applicable.
Research Design: Not applicable.
Data Collection and Analysis: Review of current practice.
Findings: The article makes transparent the lack of attention to validation of achievement tests to support inferences relevant to intended uses in instruction and project evaluation. Elements of a model for the process of reasoning from test scores are articulated. The cognitive demands on the test score user are illustrated in achievement test contexts in writing, science, and mathematics. Implications are drawn for deliberation on issues and for the development of casebooks to guide practice