1,721,003 research outputs found
Dimensionalità di alcuni indicatori di valutazione della qualità della ricerca negli atenei italiani
No full text
L’alfabetizzazione statistica degli studenti universitari. Un obiettivo raggiunto? I risultati di una ricerca condotta in Inghilterra
No full textIl contesto di studio e’ un corso introduttivo di statistica per studenti della Facolta’ di Psicologia. Due test di base, il Baseline Test A e il Baseline Test B, sono stati somministrati all’inizio del corso nella sessione accademica 2002-2003 per testare le abilita’ numeriche degli studenti. I risultati conseguiti dagli studenti in tali test di base hanno costituito oggetto specifico del menzionato articolo, dal titolo “La numeracy a livello universitario. Standard pubblici e difficolta’ concrete nei risultati di uno studio condotto in Inghilterra”. Ai due baseline test, gli studenti all’inizio del corso hanno dimostrato un vasto raggio di abilita’ numeriche. Nel complesso, la maggior parte degli studenti ha dimostrato di padroneggiare le abilita’ implicate nei compiti aritmetici e grafici di base presenti nei due test. Tuttavia, si e’ anche visto che la percentuale di studenti in grado di risolvere compiti numerici meno immediati ed elementari si riduce progressivamente al crescere del livello di complessita’ di tali compiti. Questi risultati sono stati letti come una conferma che la numeracy, pur essendo una capacita’ di base, non deve essere ne’ data per scontata a livello universitario, ne’ essere considerata obiettivo specifico o prioritario solo nei primi anni scolastici. Per di piu’, la matematica e tutte le discipline affini, come la statistica, si caratterizzano per l’essere “discipline gerarchiche”, tali per cui aree avanzate non possono essere trattate fino a che le corrispondenti aree elementari e intermedie non siano state adeguatamente coperte.
A tali due test di base hanno fatto seguito altri due test, il Follow-up test A e il Follow-up test B, somministrati al termine del corso per valutare le capacita’ di ragionamento statistico, con particolare riguardo alle capacita’ di trasferimento di conoscenze statistiche di base a situazioni e contesti reali (come ad esempio articoli tratti da quotidiani).
Il focus di questo articolo sara’ sui due Follow-up test. Per tutti gli item dei due test sono stati calcolati gli indici di difficoltà; esempi di domande facili, moderatamente difficili e difficili saranno riportati e discussi. Le implicazioni di tali risultati verranno valutate alla luce dei piu’recenti dibattiti internazionali sull’insegnamento della statistica
A classification of university courses based on students’ satisfaction: an application of a two-level mixture item response model
No full textOver the past years, Italian universities have come under increased pressure to be more competitive and attract more students, and students’ satisfaction has received increasing attention. Students’ opinions about a few aspects of academic life are sought by Italian universities in the form of a satisfaction feedback questionnaire. The aim of this paper is to classify university courses into homogeneous classes with respect to the level of students’ satisfaction through the use of a two-level mixture item response model. The data are drawn from the Italian questionnaire on students’ satisfaction administered at a Faculty of Political Sciences. The latent variables measured by the questionnaire are detected performing a model-based hierarchical clustering. Then, a special case of multilevel mixture factor model characterised by an item response parameterisation and discrete latent variables at all hierarchical levels is estimated. The study allowed us to ascertain (i) the latent dimensionality of students’ satisfac- tion with higher education courses; (ii) the varied effect of first and second-level covariates on the satisfaction dimensions; and (iii) the different sources of strength/weaknesses of the best and worst courses
Testing Unidimensionality and Differential Item Functioning of the INVALSI Students’ Data
No full textUniversity students’ skills in statistical literacy
No full textMany reports draw attention to the significant and measurable decline over the last decade in students’ mathematical skills even among those with good A-level grades in mathematics. Evidence is provided from several independent sources (e.g. QAA 2002), that even students who are well-qualified do not possess the necessary manipulative skills in algebra and basic calculus. It is argued that numeracy, a critical feature of basic mathematical ability, does not represent an elementary skill. The capability to estimate, having a sense of size and relative importance of numbers, the ability to perform simple mental arithmetic, and specific topics such as percentage and ratio must not be taken for granted at university level or seen as only the concern of the very earliest level of education.
The aim of this article is to illustrate the results of the direct assessment of students’ numeracy in a first year, introductory statistics course and the effects of basic numerical and graphical skills on students’ level of mastery of statistical concepts. Such results are part of a research project carried out at the University of Glasgow in the Academic Year 2001/2002.
Numeracy at baseline was very variable and students demonstrated a wide range of numerical skills and abilities. Also at the two Follow-up tests students demonstrated very different levels of confidence with statistical concepts. Our analysis showed that there is a top group of students who are very comfortable with a wide variety of statistical concepts. On the other hand, there is also a group of students whose level of knowledge of statistical concepts is much poorer than that of the rest of the class.
On average, Follow-up statistical knowledge was not significantly affected by students’ previous mathematical qualification but was significantly related to numeracy at both Baseline tests and Follow-up tests.
These results confirm that numeracy is a basic key skill, but it must not be taken for granted at university level. Our analysis outcomes suggest that not to require students to learn formulae or carry out calculations by hand and emphasize application of statistics to real context – which is the education approach chosen by our course – might be necessary but not sufficient to guarantee an acceptable level of understanding of basic statistical concepts. A reappraisal of the course program and a teaching approach that allows more time to be spent on fewer, more basic topics might be a possible remedy in order to insure students’ understanding of fundamental statistical notions.
Since the present research project is an observational study, the results presented can do no more than suggest a direction for future work and be considered as a general indication. We cannot claim that there is a cause/effect relationship between students’ poor numerical abilities at university entry and their subsequent difficulty in accumulating statistical knowledge, nor that numeracy is the only variable which determines the level of students’ mastery of statistical concepts. Nevertheless, we can state that there is a link between the two variables which appears to justify the concerns of those who see poor arithmetical ability as a barrier to subsequent learning.
Such results can give rise to further work in order to assess whether the outcomes found are related to the specific context of the course analysed or can be extended and considered typical features of service courses in general
Joint assessment of the latent trait dimensionality and observed differential item functioning of students’ national tests
No full textStudents’ assessment tests are routinely validated through item response theory (IRT) models which assume unidimensionality and absence of observable differential item functioning (DIF). In this paper, we investigate if such assumptions hold for two national tests administered in Italy to lower secondary school students: the Language Test and the Mathematics Test. To this aim, we rely on an extended class of multidimensional latent class IRT models characterised by: (i) a two-parameter logistic parameterisation for the conditional probability of a correct response, (ii) latent traits represented through a random vector with a discrete distribution, and (iii) the inclusion of (uniform) DIF to account for students’ gender and geographical area. A classification of the items into unidimensional groups is also proposed and represented by a dendrogram, which is obtained from a hierarchical clustering algorithm. The results provide evidence for observable DIF effects for both tests. Besides, the assumption of unidimensionality is rejected for the Language Test, whereas it is reasonable for the Mathematics Test
Composite indicators of scientific research
No full textComposite indicators (CIs) integrate a large amount of information in a format that is easily understood and are therefore a valuable tool for conveying a summary assessment of performance in priority areas. However, the construction of composite measures creates specific methodological challenges. Any CI may be considered as a model (OECD, 2008) where the CI is the response variable and the covariates are all the subjective judgements - the sources of uncertainties - which have to be made (e.g. the selection of individual indicators, the choice of normalisation methods, weighting schemes, aggregation model etc.). All these potential sources of uncertainty should be addressed because they affect both the variance of the CIs and the variability of any rankings based on CIs. In this context, sensitivity analysis can be considered as an appropriate tool to assess such uncertainties because it studies how the variation in the output can be apportioned to different sources of variation in the assumptions. Its primary aim is hence to quantify the overall uncertainty in CIs - and in country/institution rankings based on CIs - as a result of the uncertainties in the model input.
This work investigates the degree to which composite measures are an appropriate metric for evaluating and ranking the research performance of Italian universities. Do they reflect accurately the performance of universities? To what degree are they influenced by the uncertainty surrounding underlying indicators on which they are based?
The construction of composite measures creates specific methodological challenges. We address these through an analysis of some individual indicators put forward by the Italian Steering Committee for Research Evaluation (CIVR). To construct a composite indicator (CI) of scientific research, five normalisation methods, a weighting scheme, and two aggregation schemes have been computed and combined, resulting in 135 CIs. The variation in the rankings assigned by the CIs to the Universities has been explored to gauge the robustness of the CIs rankings. The analysis suggests that the judgements that have to be made in the construction of the composite can have a significant impact on the resulting score and that technical and analytical issues in the design of CIs have important policy implications
Gli indicatori compositi della ricerca possono essere considerati una misura robusta di performance scientifica degli atenei italiani?
No full textDiscussion chapter
No full textThe use of effect sizes in a keenly debated topic and one that it is seen as an important issue in educational research and evaluation for policy making. This chapter discusses the comments made at the invitational seminar, jointly organised by the Institute of Education, University of London and the National Foundation for Educational Research (NFER) on 14 November 2003 and the contributions received from the discussion forum that was available for a month after the event.
It emerged from the discussion that there is no best effect size measure. Different measures of effect size are required for different questions. However, it was also highlighted that more agreement on the best approaches to calculate effect sizes under different circumstances is needed.
Earlier on this chapter it was also indicated that relying on the p-value alone when presenting results may be inappropriate and could lead to misreporting. There was a core agreement regarding the need for practitioners and policy makers to combine value/weight/significance measures to research findings.
It was also noted that policy makers and practitioners require the output from the educational research establishment to be both clear and persuasive. Besides, caution is required against over complicated analyses without justification. However, it was also noted that simple models may need to be confirmed by more complex models which take into consideration more background factors and relationships
- …
