1,720,977 research outputs found
Determining cognitive distance between publication portfolios of evaluators and evaluees in research evaluation: an exploration of informetric methods
Full text linkThis doctoral thesis develops informetric methods for determining cognitive distance between publication portfolios of evaluators and evaluees in research evaluation. In a discipline specific research evaluation, when an expert panel evaluates research groups, it is an open question how one can determine the extent to which the panel members are in a position to evaluate the research groups. This thesis contributes to the literature by proposing six different informetric approaches to measure the match between evaluators and evaluees using their publications as a representation of their expertise.
An expert panel is specifically appointed for the research evaluation. Experts are typically selected in one of two ways: (1) straightforward selection: the person(s) in charge of the research evaluation has access to a list of acknowledged experts in specific fields, and limits its selection process to ensuring the experts’ independence regarding the program under evaluation; and (2) gradual selections: preferred profiles of experts are developed with respect to the specialization under scrutiny in the evaluation. Both ways leave some freedom for an “old boys’ network” to appoint someone without properly evaluating their qualifications. There are also other ways for expert selection, for example, inviting open application or the research groups that will be evaluated can propose their choice of experts. In research evaluation, an expert panel usually comprises independent specialists, each of which is recognized in at least one of the fields addressed by the unit under evaluation. The expertise of the panel members should be congruent with the research groups to ensure the quality and trustworthiness of the evaluation. All things being equal, panel members who are credible experts in the field are also most likely to provide valuable, relevant recommendations and suggestions that should lead to improved research quality. However, there was an absence of methods to determine the cognitive distance between evaluators and evaluees in research evaluation when we started working in July 2013.
In this thesis, we develop and test informetric methods to identify the cognitive distances between the (members of) an expert panel on the one hand, and the (whole of the) units of assessment (typically research groups) on the other. More generally, we introduce a number of methods that allow measuring cognitive distances based on publication portfolios. In academia, publications are considered key indicators of expertise that help to identify qualified or similar experts to assign papers for review, and to form an expert panel. Our main objective is to propose informetric methods to identify panel members who have closely related expertise in the research domain of the research groups based on their publications profile. The main factor that we have taken into account is the cognitive distance between an expert panel and research groups. We consider the publication portfolio of the involved researchers to reflect the position of the unit in cognitive space and, hence, to determine cognitive distance. Expressed in general terms we measure cognitive distance between units based on how often they have published in the same or similar journals. Our investigations lead to the development of new methods of expert panel composition for the research evaluation exercises.
We explore different ways of quantifying the cognitive distance between panel members and research group's publication profiles. We consider all the publications of the research groups (during the eight years preceding their evaluation) and panel members indexed in Web of Science (WoS). We pursue the investigation at two levels of aggregation: WoS subject categories (SCs) and journals. The aggregated citation relations among SCs or journals provide a matrix. From the matrix, one can construct a similarity matrix. From the similarity matrix, one can construct a global SCs or journal map in which similar SCs or journals are located more closely together. The maps can be visualized using a visualization program. During the visualization process, a multi-dimensional space is reduced to a projection in two dimensions. In this process, similar SCs or journals are positioned closer to each other.
We propose three methods, namely the use of barycenters, of similarity-adapted publication vector (SAPV) and of weighted cosine similarity (WCS). We take into account the similarity between WoS SCs and between journals, either by incorporating a similarity matrix (in the case of SAPV and WCS) or a 2-dimensional base map derived from it (in the case of barycenters). We determine the coordinates of barycenters using a 2-dimensional base map based on the publication profiles of research groups and panel members, and calculate the Euclidean distances between the barycenters. We also identify SAPV using the similarity matrix and calculated the Euclidean distances between the SAPVs. Finally, we calculate WCS using the similarity matrix. The SAPV and WCS methods use a square N-dimensional similarity matrix. Here N is equivalent to 224 WoS SCs and 10,675 journals. We used the distance/similarity between panel members and research groups as an indicator of cognitive distance. Small differences in Euclidean distances (both between barycenters and SAPVs) or in cosine similarity values bear little meaning. For this reason, we employ a bootstrapping approach in order to determine a 95% confidence interval (CI) for each distance or similarity value. If two CIs do not overlap, difference between the values is statistically significant at the 0.05 level. Although it is possible for two values to have a statistically significant difference while having overlapping CIs, the difference is less likely to have practical meaning.
Two levels of aggregation and three methods lead to six informetric approaches to quantify the cognitive distance. Our proposed approaches hold advantages over a simple comparison of publication portfolios. Our approaches quantify the cognitive distance between a research group and panel members. We also compare our proposed approaches. We examine which of the approaches best reflects the prior assignment of main assessor to each research group, how much influence the level of aggregation (journals and WoS SCs) plays, and how much the dimensionality matters. The results show that, regardless of the method used, the level of aggregation has only a minor influence, whereas the influence of the number of dimensions is substantial.
The results also show that the number of dimensions plays a major role in the case of identifying shortest cognitive distance. While the SAPV and WCS methods agree at most of cases at both the levels of aggregation the barycenter approaches yield different results. We find that the barycenter approaches score highest at both levels of aggregation to identify the previously assigned main assessor. When it comes to uniquely identifying the main assessor, all methods score better at the journal level than at the WoS SC level. Our approaches, but of course not the numerical result, are independent of the similarity matrix or map used.
All six approaches give the opportunity to assess the composition of the panel in terms of cognitive distance if one or more panel members are replaced and compare the relative contribution of each potential panel member to the panel fit as a whole, by observing the changes to the distance between the panel’s and the groups’. In addition, our approaches allow the panel composition authority to see in advance about the panel’s fit to the research groups that are going to be evaluated. Therefore, the concerned authority will have the opportunity to replace outliers among the panel members to make the panel fit well with the research groups to be evaluated. For example, the authority can find a best-fitting expert panel by replacing a more distant panel member with a potential panel member located closer to the groups
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Full text link“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Full text linkWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
UniCat library catalogue dataset
No full textThis dataset was created in the context of a project about libcitations or library catalogue analysis of book publications by Flemish Social Sciences and Humanities researchers.
The dataset was constructed on the basis of the openly available API of the Belgian UniCat library catalogue (UniCat-Search). The library catalogue was searched in September 2021 by a matching of the catalogue against the ISBNs present in the VABB-SHW database (see Aspeslagh, Peter, Guns, Raf, & Engels, Tim C.E. (2021). VABB-SHW: Dataset of Flemish Academic Bibliography for the Social Sciences and Humanities (edition 11) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5795899)
Developing a framework for a family-focused preventive intervention using the Delphi method
No full textThis paper reports on the use of the Delphi method to develop a framework for a family-focused preventive intervention. 65 knowledgeable individuals from diverse backgrounds discussed the topics and techniques of family-focused intervention, the definition of its target population, appropriate recruitment and implementation strategies, and the profile of the group leaders during the four rounds of the study. The findings illustrate the acceptability of family-focused preventive interventions in Belgian social work provided that they are culturally appropriate. The study also made a number of suggestions to support the implementation of family-focused interventions in practice. The Delphi method serves as a useful guide for the design and implementation of innovative and socio-culturally relevant interventions
Dispelling the Myths Behind First-author Citation Counts
Full text linkWe conducted a full-scale evaluative citation analysis study of scholars in the XML research field to explore just how different from each other author rankings resulting from different citation counting methods actually are, and to demonstrate the capability of emerging data and tools on the Web in supporting more realistic citation counting methods. Our results contest some common arguments for the continued
use of first-author citation counts in the evaluation of scholars, such as high correlations between author rankings by first-author citation counts and other citation
counting methods, and high costs of using more realistic citation counting methods that are not well-supported by the ISI databases. It is argued that increasingly available digital full text research papers make it possible for citation analysis studies to go beyond what the ISI databases have directly supported and to employ more
sophisticated methods
koamabayili/VECTRON-author-checklist: VECTRON author checklist
No full textWe have done our best to complete the author checklist relating to the use of animals in the hut study. Note that the objective for the hut study was to evaluate the IRS treatment applications for residual efficacy against Anopheles mosquitoes, including the local An. coluzzii mosquito population. Cows were only used to attract mosquitoes into the huts and no tests were carried out directly on the cows. The author checklist is intended for use with studies where experiments are carried out on animals, which is why we have had such difficulty in completing this for the hut study, as many of the questions do not relate to how the cows were used
- …
