8 research outputs found
The Effects of Testlets on Reliability and Differential Item Functioning
No full textReliability and differential item functioning (DIF) analyses were conducted on testlets displaying local item dependence in this study. The data set employed in the research was obtained from the answers given by 1500 students to the 20 items included in six testlets given in English Proficiency Exam by the School of Foreign Languages of a state University in Turkey. One of the purposes of this study was to determine the influences of the tests composed of testlets on reliability, so the reliability coefficients obtained for cases where the influences of testlets were considered and those for cases where the testlet influences were not considered were compared. In consequence of the G theory analyses conducted in this context, it was found that the G and Phi coefficients estimated by not considering the testlet effects were higher than those estimated by considering the testlet effects. It was concluded that the reliability was estimated to be relatively higher when the influences of the testlet were not considered. Two methods were used in this study so as to determine the effects of testlets on differential item functioning and the results were compared. In the DIF-determining method considering the testlet effect, both the number of items displaying DIF at the significant and estimated levels of DIF were found to be higher than in the method not considering the testlet effect.WoSScopu
Coping with Unbalanced Designs of Generalizability Theory: G String V
No full textThe aim of this paper is to introduce a software that is appropriate for the generalizability theory for not only balanced but also unbalanced data sets. Because it is possible to have unbalanced data sets while conducting a study, the researchers have devised an easy solution, other than deleting data, to balance the design to cope with this situation. Thus, the software G String V will be introduced. First, the generalizability theory will be reviewed, followed by a description of the unbalanced synthetic data that was used to conduct the analysis using the software. Explanations are provided for installing the software, preparation of the data, and the step-by-step data analysis. Moreover, the interpretation of the data is also explained. Finally, the limitations of the software are shared.Wo
Effects of Category Labeling With Emojis on Likert-Type Scales on the Psychometric Properties of Measurements
No full textToday, emoji have become a popular option for anchoring the categories of Likert-type scales applied to not only adults but also children. The aim of this study was to investigate the effects of category labeling with emoji by comparing the psychometric properties of the emoji- and verbal-anchored versions of the mathematics motivation scale applied to students aged 8-11 years (grades 3, 4, and 5). The participants of the study comprised 658 students. According to the research results, students used the upper categories of the scale more intensively in the emoji-anchored version than in the verbal-anchored one. Parallel to this, the means calculated for the emoji-anchored version were found to be significantly higher. Moreover, the results of the research revealed that the verbal-anchored scale form yielded more reliable and valid measures than the emoji-anchored version
Identifying the Relationship between the Reading Literacy and Learning Strategies According to PISA 2009
No full textThe purpose of this study is to investigate the learning strategies that affect reading literacy of 15-year-old Turkish students in Programme for International Student Assessment (PISA). Elaboration, control, and memorization strategies are defined as learning strategies. These variables are used to conduct structural equation modeling. In the study, the designed model's goodness of fit indexes are good enough according to standard values of fit indexes. According to research results, all the relations between elaboration, control, and memorization strategies and reading literacy are meaningful. The findings of the study indicate that 30% of the reading literacy could be explained by these observed variables
Thematic Content Analysis of Studies Using Generalizability Theory
No full textOne of the important theories in education and psychology is Generalizability (G) Theory and various properties distinguish it from the other measurement theories. To better understand methodological trends of G theory, a thematic content analysis was conducted. This study analyzes the studies using generalizability theory in the field of education in Turkey by using the method of thematic content analysis. It reviews 60 studies, including 31 articles and 29 theses published from 2004 to 2017. The selected studies underwent thematic content analysis using parameters including tagged information, aim, G Theory type, number of facets used in the study, Turkish word for “facet,” object of measurement, sample size, design type, mixed-design availability, shared results of G and D studies, computer programs, method of calculating negative variance, availability of fixed facets, and design balance. The data were interpreted on the basis of frequencies; both table and figures are included in the study. According to the results, there is an increase in the number of studies conducted by using G theory by years. Of these, many compare theories; most of them applying univariate G Theory and consider two-faceted measurement situations. While a small subset of studies features mixed design, a large group features crossed design, with individuals as the object of measurement. The computer program most commonly used in analyses is EduG. The majority of studies use balanced design. Recommendations are provided accordingly with the results.Wo
The Turkish Adaptation of the Statistics Anxiety Scale for Graduate Students
No full textIn this study, it was aimed to adapt the Statistical Anxiety Scale (SAS) developed for graduate students by Faber, Drexler, Stappert and Eichhorn to Turkish. The research was carried out on 375 students attending graduate education in any field in Turkey. In the study, construct validity of the SAS was investigated via exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Parallel analysis method was also used in making decision about factor number of the scale. In the EFA and parallel analysis, a unidimensional structure was obtained in line with the results acquired in the factor analysis of the original form of the SAS. However; since the original form of the SAS was designed by foreseeing a three-dimensional structure of worry, avoidance and emotionality, both unidimensional and three-dimensional structures were tested in CFA. The fit indices reported in CFA were found to be within acceptable limits for both models. In the reliability analysis, Cronbach Alpha internal consistency coefficient was calculated as .91 for the whole scale, and it was found to be .91, .83, and .91 for worry, avoidance and emotionality dimensions, respectively. It was determined that item correlations exceed the lower limit of .30 for all items in the scale. Ferguson Delta statistic, which provide evidence for the discriminatory power of the entire scale, was determined as .98. These results suggest that the Turkish form of the SAS yields valid and reliable measures.Wo
Developing a virtual patient: design, usability, and learning effect in communication skills training
No full textAbstract Background Literature shows that Virtual Patients (VPs) find extensive usage in the field of health sciences, especially in the post-pandemic period. VPs are successfully utilized in developing various effective skills like medical interview. However, this technology is quite new in Turkey and has not yet been used in communication skills training in a structured form. This research aimed to develop a virtual patient to improve the communication skills of medical students. Methods Developmental research method was used in the study. The implementation phase involved the one group posttest quasi-experimental design. The study group comprised of experts in various fields and 213 medical students. Needs Analysis Form, Scenario Building Form, System Validation Form, Communication Skills Assessment Form, and Interview Form were used as data collection tools. The research primarily concentrated on ensuring minimal errors within the system and enhancing students’ communication skill scores. Results The study found that VP was effective in teaching communication skills. Communication skills improved from a mean score of 36.74 in the first interview with 15 students to 74.2 in the final application with 198 students. It was determined that the students who practiced repeatedly (n = 26) made 17% more effective interviews than their first practices (score: 89.2). The script matching of the VP was 83%. Other data obtained from the students generally showed that the VP application was developed in accordance with the purpose, that it was user-friendly, and that the scenarios were adequate. Conclusion VPs like this have the potential to develop skills such as history taking, clinical reasoning, etc., which are very important in the field of health sciences
Automated Assessment and Feedback Virtual Patient Chatbot for Clinical Anamnesis Training
No full textLa comunicación efectiva entre médicos y pacientes en la anamnesis clínica es una competencia esencial en la educación médica. Para su desarrollo, la evaluación y retroalimentación por parte de expertos resultan fundamentales para un aprendizaje efectivo. Sin embargo, los métodos tradicionales, como las entrevistas simuladas con pacientes actores o pacientes estandarizados (SPs), presentan limitaciones en términos de inmediatez, escalabilidad y objetividad. La evaluación en estos enfoques es susceptible a la subjetividad de cada evaluador. Ademas dependen de la revisión manual por parte de docentes, lo que puede generar demoras, o de la retroalimentación de los SPs, cuyo uso implica altos costos. Este estudio propone la optimización de un Paciente Virtual basado en Modelos de Lenguaje de Gran Escala (LLMs) con un módulo de evaluación automática que permita analizar, calificar y proporcionar retroalimentación sobre entrevistas clínicas simuladas.Maestrí
