Journal of MultiDisciplinary Evaluation (JMDE)
Not a member yet
528 research outputs found
Sort by
Review of Social Network Analysis in Program Evaluation
Get PDFThe fall, 2005, New Directions for Evaluation journal included nine articles dedicated to social network analysis (SNA) in Program Evaluation. Four of the nine articles in the journal explored the academics of SNA methodology, while the other five articles presented examples of quite diverse applications of SNA in program evaluation. This review will outline the major arguments for the use of SNA, applications of SNA in program evaluation, and a critique of the SNA content contained in this journal
Review of New Directions for Evaluation, Volume 106: Theorists' Models in Action
Get PDFThe summer 2005 issue of New Directions for Evaluation, “Theorists’ Models in Action”, edited by Marvin C. Alkin and Christina A. Christie is, in part, inspired by the “Radnor Middle School” case study conducted in the early 1980s (Brandt, 1981). In the Radnor case study notable theorists including Scriven, Stake, Eisner, and Popham were asked to explain how they would evaluate Radnor’s humanities curriculum.
In this issue Alkin and Christie asked four contemporary theorists—Jennifer C. Greene, Gary T. Henry, Stewart I. Donaldson, and Jean A. King—to describe how they would evaluate the case of the “Bunche-Da Vinci Learning Partnership Academy.” The Bunche-Da Vinci Learning Partnership Academy is essentially a “unique partnership between the [school] district and a nonprofit educational company specializing in innovative school interventions for low-performing students” (Eisenberg, Winters, & Alkin, 2005, p. 5)
Planning and Evaluation: Two Sides of the Same Coin
Get PDFThis article outlines the principles, tools, and practices of Project Cycle Management (PCM), a methodological approach integrating planning, implementation, monitoring, and evaluation. PCM is aimed at improving the success of development projects by creating sustainable benefits for target groups
Using Culturally Sensitive Methodologies When Researching Diverse Cultures
Get PDFThis article introduces additional information in the ongoing conversation about using culturally sensitive research methods with diverse research populations. Research, including evaluation research, examining ethnic minorities, international, tribal members, and individuals within diverse cultures should be performed within a context of cultural understanding. Several methodological examples will be presented, expanding the discussion of contemporary research with diverse cultures
The Transdisciplinary Model of Evaluation
Get PDFThe transdisciplinary view, or model, of evaluation requires an understanding of how and why evaluation developed from a practice to a highly skilled, professional practice to a field-specific discipline, and finally to an autonomous discipline and transdiscipline, much like ethics, statistics, and measurement (Scriven, 2003). This understanding becomes known from the transdisciplinary model’s three primary characteristics that make it a transdiscipline, which are: epistemological;political; and disciplinary (Scriven, 1993)
The Evaluation of Research Merit versus the Evaluation of Research Funding
No full textThe evaluation of research and researchers is an example of a fairly basic kind of evaluation. It normally involves either a ranking or a grading (aka., rating) of research projects or personnel for merit, worth, or significance, and these are tasks that we know a good deal about doing.
But the evaluation of research funding is another kind of animal altogether. It aims for an apportionment or allocation decision, which is either something essentially different from evaluation or, with a stretch, a highly complex kind of evaluation decision. It is certainly a decision that depends on more than one kind of basic evaluation, but it depends on them in a way that has never been reduced to a formula or computer program
In This Issue
Get PDFIn this issue, we have some methodological reflections from another valorous (female) evaluator working successfully on the HIV/AIDS front in the darkest reaches of the Amazon jungle, with a largely law-breaking clientele whose general attitude about any breach of confidentiality seems to have been that it would probably cause their death, so why not yours as well. The article is a kind of handbook of hazards for evaluators working ‘in harm’s way’ and hence a good guide to planning for others. Interestingly, it may provide the strongest argument for empowerment evaluation: do it when it’s the only possibility
Aotearoa New Zealand Evaluation Association (anzea) is Formed
Get PDFAn announcement is made regarding the creation of anzea and its first conference in 2007
Using Principal Components Analysis in Program Evaluation: Some Practical Considerations
Get PDFPrincipal Components Analysis (PCA) is widely used by behavioral science researchers to assess the dimensional structure of data and for data reduction purposes. Despite the wide array of analytic choices available, many who employ this method continue to rely exclusively on the default options recommended in dominant statistical packages. This paper examines alternative analytic strategies to guide interpretation of PCA results that expand on these default options, including (a) rules for retaining factors or components and (b) rotation strategies. Conventional wisdom related to the interpretation of pattern/structure coefficients also is challenged. Finally, the use of principal component scores in subsequent analyses is explored. A small set of actual data is used to facilitate illustrations and discussion.
Despite the increasing popularity of confirmatory factor analytic (CFA) techniques, principal components analysis (PCA) continues to enjoy widespread use (Kellow, 2004; Thompson, 2004). Researchers who employ PCA are typically interested in (a) assessing the dimensional structure of a dataset (Dunteman, 1989) or (b) reducing a large number of variables into a smaller set of linear combinations (components) for subsequent analyses (e.g., multiple regression). For instance, an evaluator may have occasion to develop a new instrument and wish to ascertain the number and features of the underlying dimensions represented in the data. At other times an existing measure is modified or shortened and the sample data are used to explore the extent to which the structure of the original version has or has not been substantively altered (although CFA is a stronger method for this purpose). The PCA approach also is useful for creating new variables that are linear combinations of a set of highly correlated original variables. These new composite variables may then be used in subsequent analyses. As Stephens (1992) notes, “... if there are 30 variables (whether predictors or items), we are undoubtedly not measuring 30 different constructs, hence, it makes sense to find some variable reduction scheme that will indicate how the variables cluster or hang together” (p. 374). Use of PCA helps to solve at least two problems. First, the presence of multicollinearity (high inter-item or variable correlations) leads to inflated standard errors for the measured variables when conducting statistical analyses, which increases the probability of Type II errors (non-significance when a significant difference exists in the population). Second, when one is using a large set of variables to predict or explain another variable (or set of variables) as opposed to a smaller set of composites, one pays a price in terms of the degrees of freedom used in the analysis. All other things being equal, the more degrees of freedom expended the smaller the value of the omnibus test statistic (e.g., F) that results from the analysis (Stephens, 1992).
There are a number of important issues related to the data in hand that need to be addressed (e.g., linearity; absence of outliers) before invoking PCA, and readers are referred to Tabachnick and Fidell (2001) for an excellent overview of these considerations. Once PCA is determined to be appropriate, the analysis proceeds in a series of sequential steps―several options are available to researchers at each step. Too often researchers rely on the default options provided in the major statistical packages and fail to examine other options that may allow for fuller exploitation of the data. The purpose of the present paper is to briefly explore the options available to analysts with respect to (a) rules for retaining principal components and (c) rotation strategies. In addition, conventional wisdom related to the interpretation of pattern/structure coefficients is challenged on substantive grounds. Finally, we briefly explore how PCA may be used to derive component scores for further data analysis