201 research outputs found
Portrait of Professor David Andrich
No full textPortrait of Professor David Andrich, Dean, School of Education at Murdoch University.
The complete set of 4 negatives is available at the Murdoch University Library.
These images are part of the History of Murdoch University Collection
Maintaining a common arbitrary unit in social measurement
No full textIn educational assessment, it is common to equate test forms in order to draw comparisons between different populations of students. The process of test equating presents a number of challenges, many of which relate inherently to the problem of maintaining a common unit and origin.
In order to develop a general theoretical approach to maintaining a common unit and origin in the measurement of quantitative attributes, the role of the unit is carefully examined. Classical physics is explicitly adopted as the guiding paradigm during the investigations throughout the dissertation. Accordingly, the central objective is to develop a theoretical foundation for maintaining a common unit and origin which meets two criteria: (i) it must be congruent with the definition of measurement in physics captured in the classical theory of measurement (Michell, 1999); and (ii) it must meet a key requirement of measurement in the physical sciences identified by Rasch (1960/1980). Rasch identified the relevant requirement, that of invariant comparison, based on analysis of Newton's second law and showed that the Principle of Invariant Comparison is formally embodied in his measuring function for dichotomous data (Rasch, 1960/1980). This model provides the basis for the development and exposition of general concepts and principles in the dissertation.
In order to achieve the central objective, the unit is made formally explicit and specified in relation to the experimental frame of reference. Rasch (1977) defined a Specified Frame of Reference (SFR) in terms of a collection of objects, a collection of agents, and outcomes of the interaction between these. Drawing on a fundamental distinction introduced by Andrich (2003), the unit of a SFR is referred to as a natural unit and is distinguished from an arbitrary unit, the magnitude of which is theoretically independent of any particular SFR and instrument contained within. From this distinction, a definition of discrimination arises naturally; a definition that is also congruent with classical physics. The distinction and related definitions provide the basis for derivation of a general form of Rasch's measuring function for dichotomous data, referred to as the Extended Frame of Reference Model (EFRM). It is shown that the EFRM provides a rational basis for maintaining a common unit and origin in assessment contexts involving two or more Specified Frames of Reference.
Simulation and empirical studies are employed to illustrate application of the EFRM. These studies also serve to illustrate that quantitative hypotheses entailed by the EFRM are open to empirical tests by providing a context for the use of graphical methods and statistical tests of fit. Empirical investigations are used to illustrate consequences of differences between natural units in the context of applied educational assessment. The studies also provide a context in which to characterise the model, and the structure of data that it entails. Although the simulation studies demonstrate the basic efficacy of the model, they also indicate scope for improvement in terms of the precision of estimates. To explore possible approaches to refining the estimation process, Maximum Likelihood (ML) equations are derived and examined. Firstly, Joint Maximum Likelihood (JML) equations are presented. Following this, Conditional Maximum Likelihood (CML) equations are derived. It is shown that while the CML equations permit separation of the person and item parameters, item locations are expressed in terms of natural, rather than arbitrary, units. A particular approach is proposed, emphasising links to the classical theory and the Principle of Invariant Comparison. In considering the proposed approach, a distinguishing feature of the definition of discrimination is highlighted: specifically, the nature of its definition represents the importance of relationships between quantitative attributes, and the specific structure of these relationships, to the measurement of any particular attribute. Although it is not possible to fully study this feature given the scope of the work, it is a key to the implications of the general theoretical framework embodied in the EFRM. Accordingly, these implications are touched on before concluding the dissertation
Measuring youth civic development in Malaysia: Conceptualization, instrument development using the Rasch measurement model, and substantive outcomes
No full textThis study concerned the development of measures for youth civic development in Malaysia with four major goals in mind, namely conceptual, measurement, baseline/pragmatic, and predictive goals. It was a preliminary study of its kind in Malaysia as Malaysia did not participate in any international or regional study such as the IEA Civic Study. The central intention of the study was to establish a conceptually valid assessment framework and psychometrically sound instruments for the measurement of youth civic development in Malaysia. This would, in turn, provide some baseline information upon which future research on youth civic development in Malaysia could build.
Civics as a subject was removed from the Malaysian school curriculum in the early 1980s and Civic and Citizenship Education as a specific subject, with an explicit structure and curriculum framework, was reintroduced only in 2005. It was not feasible at the time when this study was conducted to conduct a specific assessment on the outcomes for school-aged students as explicated and expected in the curriculum framework. Therefore, the post-school-aged group of young undergraduates in public universities within the age range of 17-28 was selected as the target population.
Firstly, a conceptualisation of youth civic development was synthesized by identifying current views on citizenship, as set out in the international literature and, specifically, in Malaysia in two major documents – the Rukenagara and Vision2020. Civic development consists of three variables of Civic Knowledge (CK) (knowledge about the legal status and associated rights and responsibilities of citizens); Civic Disposition (CD) (views on identity and attitudes as a citizen); and Civic Engagement (CE) (participation as a citizen). The relationship between civic development variables was explicated through a Neo-Vygotskian cultural-historical theory of human development. In addition, Bronfenbrenner’s Socio-Ecological Theory of human development was adopted for the selection of contextual and individual factors (the independent variables) for the conceptual model. To help ensure cross-cultural validity for use within the Malaysian socio-cultural context, the three dependent variables of CK, CD and CE, and the independent variables were appraised for their suitability for use within the Malaysian socio-cultural context before they were operationalized into observable indicators.
Secondly, the Rasch measurement paradigm, framework and model were adopted as the foundation for instrument development and validation. The procedures for instrument development followed Wilson’s model of four building blocks of instrument development (Wilson, 2005). Three instruments were developed for this study, namely The Malaysian Civic Knowledge Inventory (MCKI), The Malaysian Civic Disposition Inventory (MCDI), and the Malaysian Civic Engagement Inventory (MCEI). Items from existing instruments in international literature were adopted or adapted for the three instruments, especially the MCEI, if there was construct equivalence in Rukunegara and Vision 2020. This was to ensure a cumulative tradition in research on youth civic development. Most of the items in the MCKI and the MCDI however were developed specifically for this study to capture the specificity of the civic culture of Malaysia.
The target population for the study was undergraduate students enrolled in Malaysian public universities. Students at one public university were chosen as the accessible population. Data were collected in February 2006 (pilot study) and from July to August 2006 (main study). The sample for this study (N=1391) was drawn through multistage cluster sampling by study concentration, level of study, and clusters based on lecture/tutorial group. The sample also, by default, included major inherent characteristics of the target population, particularly gender (Male and Female) and ethnicity (Malay, Chinese, Indian and other ethnic groups).
Rasch analyses confirmed the three-dimensional structure of youth civic development. Three measurement scales with acceptable psychometric properties were established to provide measures for the three dependent variables of CK, CD and CE. The hypotheses about civic development (in terms of CK, CD and CE) as a ‘tool and result’ activity were tested through path analyses of mediational relationships based on Baron and Kenny’s criteria (1986) using the Rasch-derived linearized scores from the main study. Analyses provided statistical support for a bidirectional association between each pair of CK, CD and CE, despite the low inter-correlations between them.
This study revealed that, on average, youth in this study demonstrated a moderately high level of CK (Mean =1.11, SD = 0.77), a positive CD (Mean = 1.25; SD = 0.63) but a moderately low CE (Mean = -0.44; SD = 0.92). The Malay sub-sample scored, on average, higher on all three dimensions of civic development. The results of hierarchical multiple regressions however showed the effect of ethnicity (Malay or Non-Malay) was statistically non-significant when other contextual (home, curricular and co-curricular) variables were entered into the regression equation. This indicates it is not ethnicity that predicts levels of civic development, but rather it is the differences in other socio-political entitlements and status associated with ethnic status. Three selected collective social-contextual factors of home, curriculum and co-curriculum explained only a fairly modest but statistically significant amount of variance (10 to 20%) in the dependent variables.
Finally, the findings were discussed in relation to the theoretical perspectives undergirding this study. Pragmatic implications for policy planning as well as other relevant stakeholders involved in youth civic development are also discussed
Developing critical numeracy at the tertiary level
No full textStudents at university encounter quantitative information in tables and graphs or through prose in textbooks, journals, electronic sources and in lectures. The degree to which students are able to engage with this kind of information and draw their own conclusions, influences the extent to which they need to rely on the interpretation of others. In particular, students who are studying in non-mathematical disciplines often fail to engage seriously with such material for a number of reasons. These may include a lack of confidence in their ability to do mathematics, a lack of mathematical skills required to understand the data, or a lack of an awareness of the importance of being able to read and interpret the data for themselves. In this thesis, the successful choice and use of skills to interpret quantitative information is referred to as numeracy.
The level of numeracy exhibited by a student can vary depending on the social or cultural context, his/her confidence to engage with the quantitative information, the sophistication of the mathematics required, and his/her ability to evaluate the findings. The first part of the thesis is devoted to the conceptualisation of numeracy and its relationship to mathematics.
The empirical study that follows this is focused on an aspect of numeracy of importance to university students: the reading and interpreting of tables of data in a range of non-mathematical contexts. The students who participated in this study were enrolled in degree programs in the social sciences. The study was designed to measure the effectiveness of a one-hour intervention workshop aimed at improving the levels of the students? numeracy. The short length of the intervention was dictated by practical and organisational constraints. This workshop involved reading and interpreting a table of data using strategies based on the SOLO taxonomy (Biggs and Collis, 1982).
The SOLO taxonomy was developed mainly as a means of classifying the quality of responses across both arts and science disciplines. The categorisation uses five levels: prestructural, unistructural, multistructural, relational and extended abstract. It can be used as a diagnostic tool at all levels of education as it can be seen as a spiral learning structure repeating itself with increasing levels of abstraction. It can also be used as a teaching tool in feedback to students.
A measuring instrument, also based on the SOLO taxonomy, was designed to gauge the levels of the students' responses to these tasks. Each response was allocated a level that was subsequently coded as a number from zero to seven. Because the responses were in distinct ordered categories, it was possible to analyse the scores using the Rasch Model (Rasch 1960/80) for polytomous responses, placing both the difficulty of the tasks and the ability of the students on an equal interval scale. The Rasch Model was also used to evaluate the measuring instrument itself. Some adjustments were made to the instrument in the light of this analysis. It was found that it is possible to construct an instrument to distinguish between levels of students' written responses for each of the chosen table interpretation tasks.
The workshop was evaluated through a comparison of the levels achieved by individual students before and after the workshop. T-tests for dependent samples indicated a significant improvement (p < 0.01) in student performance
Selecting medical students: an Australian case study
No full textThe recognition that medical practitioners require more than simply a high level of academic ability to function successfully in their profession, together with a sharp increase in the number of academically qualified applicants to medical courses, has led to new ways of selecting medical students. Consequently the selection of students into the high-stakes course of medicine has become an area of considerable interest and research activity. The issues involved in selection are now prominent in the medical and medical education literature published in the UK, the USA, Australia, New Zealand and Canada, and in some European countries. At the same time as the introduction of new selection procedures, and independently of it, due to advances in pedagogy the nature of the medical curriculum has also changed. Changes have been characterised by the use of problem-based learning, and an emphasis on self-directed learning, as well as an increase in interaction between the students in classes and between students and their teachers. The recognition that problemsolving, communication and interaction skills in the courses, in addition to the requisite intellectual capacity, would enhance performance as practitioners, has reinforced the need for students to be selected on a different set of attributes from those used previously.
In Australia, changes in the way in which medical students are selected were initiated by the University of Newcastle in the early 1990s, with the introduction of some tests of cognitive skills and an interview. Over the following ten years, the other Australian undergraduate medical schools followed suit and a three-component selection process developed in an attempt to differentiate among the high calibre applicants to medical courses, by identifying additional important skills and attributes. The three components are the academic score, results on an aptitude test and results on a selection interview. Two of the nine undergraduate medical schools chose not to use a selection interview.
The focus of this thesis is on evaluating the new selection processes and investigating the consequences of the admission of school-leaver applicants into undergraduate medical courses, where the Tertiary Entrance Rank (TER) is the academic criterion for determining suitability to undertake tertiary studies. Each undergraduate medical school has developed its own unique way to operationalise the selection of its students. However, the use of the Undergraduate Medicine and Health Sciences Admissions Test (UMAT), which developed out of the University of Newcastle’s test of cognitive skills, and the conduct of an interview for a select group of applicants are common to these practices. The implementation of the new selection processes has not been without its critics, mainly from within the medical profession. This thesis studies the issues which underlie the three components of selection (the TER, the UMAT and a selection interview) and uses as a case study the particular process used to select students into the six year undergraduate medical course at The University of Western Australia (UWA).
The UWA selection process involves applicants passing a threshold score on each of the three components and then being ranked by a mechanism which combines the three scores with equal weight. This is a compensatory system in which applicants can compensate for a score near the threshold on one component by high scores on the other two components. This study showed that the resultant cohort is eclectic in its characteristics, with the full range of scores (above the threshold) in each component being represented.
Both qualitative and quantitative methods of data collection were used to address the issues surrounding the way in which medical students are selected and the outcomes of such processes. First, semi-structured interviews were held with different groups of stakeholders, including the staff at secondary schools which prepare the students for tertiary entrance; academic and administrative staff at The University of Western Australia (UWA); academic and administrative staff at other Australian universities; and senior staff at the major teaching hospitals in Western Australia. Secondly, quantitative studies on UWA data addressed the predictive validity of the components of selection; inter-rater reliability and the internal consistency of the data sets from the selection interview; and the attrition rate in the course.
Outcomes from the research showed that in general, the reactions from stakeholders have been positive. Importantly, academic levels amongst medical students and recent graduates do not appear to have been eroded by the new process, in which the academic threshold has been lowered. The UMAT is a contentious national test which has had its validity as a selection instrument questioned. A recent construct and content validity study on the UMAT (Mercer and Chiavaroli, 2006) has gone some way towards settling some of these issues, but the question of predictive validity has yet to be addressed adequately. The existence of commercially available preparation courses has been controversial because of the equity issues involved for those applicants who for some reason do not have access to such courses.
The selection interview, one of the three components of selection, conducted by the Faculty of Medicine, Dentistry and Health Sciences at UWA, whilst attracting criticism from some for appearing stilted and overly prescriptive, was judged to be robust and rigorous by many of those directly involved in its implementation. Furthermore a high proportion of medical students were judged by their teachers to have good communication skills, which is a positive outcome for future members of the profession. A study to quantify reliability indices for the UWA selection interviews indicated high levels of inter-rater reliability and internal consistency of the ratings data produced.
The predictive validity study conducted as part of this study showed the two major predictors of course outcomes at UWA to be the TER and female gender. The TER predicts outcomes in the knowledge-based units across the course and in some clinically-based units in the later years. However, the interview score (in particular the Communication Skills component) and scores on the first section of UMAT (Logical reasoning and problem solving) also predict outcomes in some of the clinically-based units.
The results of these studies are encouraging to those who believe that the new selection process, whilst imperfect, has gone some way towards solving the problems attached to selection based solely on academic merit. The question now becomes how to improve further on the selection of medical students and to do so in an evidence-based way. The characteristics to be included in selection remain controversial. The rigorous assessment of such characteristics needs to be addressed in the longer term and will be an evolving issue, as the medical curriculum and the nature of the profession also continue to change
Intellectual development of adolescents and pre-adolescents from psychometric and cognitive developmental perspectives, 1985-1988: File rs3
No full textThe aim of this study was to conduct a part longitudinal and part cross-sectional study of the rate of growth of intellectual capacity of individuals, and in particular to determine: the nature of the development on a test set in the psychometric tradition and scaled to subscribe to the fundamental measurement; the relationship of this development to development ascertained from the cognitive developmental tradition associated with the work of Piaget; and the relationship of cognitive development in what are termed the social and physical domains. An extended test analogous to the Raven's (ACER 1958) Progressive Matrices Tests were used. Variables coded in each data set include gender and scored responses to the Raven's items administered at each testing session. The final data file contains information regarding the initial age of the subjects and their heights across all 6 testing sessions
Intellectual development of adolescents and pre-adolescents from psychometric and cognitive developmental perspectives, 1985-1988: File rs1
No full textThe aim of this study was to conduct a part longitudinal and part cross-sectional study of the rate of growth of intellectual capacity of individuals, and in particular to determine: the nature of the development on a test set in the psychometric tradition and scaled to subscribe to the fundamental measurement; the relationship of this development to development ascertained from the cognitive developmental tradition associated with the work of Piaget; and the relationship of cognitive development in what are termed the social and physical domains. An extended test analogous to the Raven's (ACER 1958) Progressive Matrices Tests were used. Variables coded in each data set include gender and scored responses to the Raven's items administered at each testing session. The final data file contains information regarding the initial age of the subjects and their heights across all 6 testing sessions
Intellectual development of adolescents and pre-adolescents from psychometric and cognitive developmental perspectives, 1985-1988: File rs4
No full textThe aim of this study was to conduct a part longitudinal and part cross-sectional study of the rate of growth of intellectual capacity of individuals, and in particular to determine: the nature of the development on a test set in the psychometric tradition and scaled to subscribe to the fundamental measurement; the relationship of this development to development ascertained from the cognitive developmental tradition associated with the work of Piaget; and the relationship of cognitive development in what are termed the social and physical domains. An extended test analogous to the Raven's (ACER 1958) Progressive Matrices Tests were used. Variables coded in each data set include gender and scored responses to the Raven's items administered at each testing session. The final data file contains information regarding the initial age of the subjects and their heights across all 6 testing sessions
Intellectual development of adolescents and pre-adolescents from psychometric and cognitive developmental perspectives, 1985-1988: File rs5
No full textThe aim of this study was to conduct a part longitudinal and part cross-sectional study of the rate of growth of intellectual capacity of individuals, and in particular to determine: the nature of the development on a test set in the psychometric tradition and scaled to subscribe to the fundamental measurement; the relationship of this development to development ascertained from the cognitive developmental tradition associated with the work of Piaget; and the relationship of cognitive development in what are termed the social and physical domains. An extended test analogous to the Raven's (ACER 1958) Progressive Matrices Tests were used. Variables coded in each data set include gender and scored responses to the Raven's items administered at each testing session. The final data file contains information regarding the initial age of the subjects and their heights across all 6 testing sessions
A probabilistic IRT model for unfolding preference data
No full textA probabilistic model is developed for the pair-comparison
design in which the unfolding principle
that governs the choice process uses a discriminal process
analogous to Thurstone’s Law of Comparative
Judgment. However, this process is governed by the
square of the distance between the location of the person
and the stimulus, rather than controlled by the location
of the stimulus as in Thurstone’s formulation.
A simulation study demonstrates the feasibility of estimation,
and two examples use real data to show the
implications of the unfolding models for psychological
research. Index terms: choice data, item response
theory, pair comparisons, preference data, unfolding.Andrich, David. (1989). A probabilistic IRT model for unfolding preference data. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/107307
- …
