1,720,971 research outputs found
Investigating the impact of technologies on the quality of data collected through surveys
Get PDFSocial surveys continue to play an important role in social science and policy making. In addition to providing information about attitudes and behaviours, they act as a vehicle for the collection of many different types such as biomeasures, geographical data, administrative records, social media posts and so on. The resilience and adaptability of the social survey owes much to the way that they have adapted to the enormous changes in the technological environment. Radical changes in telephony, personal computing, the internet, and mobile devices have transformed many aspects of the research process. While these changes have brought many benefits, the application of each new technology in survey data collection needs careful consideration in terms of ethics, burden, cost, and implementation. Moreover, they may introduce representation errors and measurement errors that must be accounted for. In this context, this thesis considers the effects of new technologies on data quality and measurement error. It presents three examples of the use of technologies in social surveys and examines an aspect of data quality in relation to each. The thesis makes specific recommendations and encourages further methodological research in the use of technology in survey data collection.The focus of the first study is on three biomeasures which are frequently collected in health or multidisciplinary surveys but may be recorded using different equipment. A randomised cross-over trial of 118 healthy adults aged 45-74 years was conducted using two sphygmomanometers to measure blood pressure, four handgrip dynamometers to measure grip strength, and two spirometers to measure lung function. For each of these three measures, multiple readings from each device were combined with information about the individual, drawn from a self-completion questionnaire, to build a pseudo-anonymised analytical dataset. Evidence was found of differences in measurements when assessed using alternative devices. For blood pressure, there is a difference, on average, of 3.85 mm Hg for Systolic Blood Pressure and 1.35 mm Hg for Diastolic Blood Pressure. For grip strength, two electronic dynamometers record measurementson average 4-5kg higher than either a hydraulic or a spring-gauge dynamometer. For lung function, a difference of 0.47 litres, on average, was found for measures of Forced Vital Capacity, but no difference was found in measures of Forced Expiratory Volume. The primary analysis was conducted using Bland and Altman plots. Sensitivity analyses tested different definitions of each measure and used multilevel regression modelling as an alternative way of estimating device effects. The findings have implications for analysts who may want to test the sensitivity of their findings to the average differences observed with these combinations of devices and may help investigators who are selecting equipment for new studies or changing equipment for longitudinal studies. Further trials are needed to replicate the comparison of these devices and to test different device combinations, both in stand-alone studies and within larger observational surveys. Future analysts may wish to consider using multilevel modelling to assess device effects.The second paper also considers device effects, this time, exploring whether the device used to complete an online survey (that is a PC, smartphone, or tablet) affects data quality. The study is based on the Wellcome Trust Science Education Tracker, a mobile-optimised, online survey of over 4,000 pupils aged 14-18. It uses the Wellcome Science Education Tracker 2016 dataset, available through the UK Data Archive, with additional survey process variables obtained with the agreement of the Wellcome Trust. The study uses propensity scores (more specifically, Inverse Probability Treatment Weights) to balance the samples, to reduce the possibility that measurement effects are confounded by selection. The analysis draws on linked geographical, administrative and survey process data which provides an opportunity to assess the use of exogenous confounder variables in the matching process. The large sample size makes it possible to test the sensitivity of the finding to the inclusion or exclusion of tablet users. Overall, the study identifies few consistent device effects, and those that are observed are small, providing reassurance for survey practitioners and analysts. After controlling for selection, those who use a mobile device are seen to have higher levels of “don’t know” responses and are more likely to have interruptions during survey completion. Contrary to the findings of some earlier studies, smartphone responders complete the survey more quickly than PC responders. The results for straightlining are mixed and no clear pattern between mobile and PC could be found. The findings encourage the inclusion of a wide range of covariates when controlling for selection, beyond basic demographics, ideally including exogenous variables, and including those which capture topic salience.The third research study addresses the potential for app-based research. It is an exploratory study which assesses the quality of data collected using an app-based expenditure diary over a one-month period. A total of 268 members of the Understanding Society Innovation Panel agreed to take part. The analysis uses a combination of two datasets from Understanding Society: Spending Study 1 (2016-2017) and Wave 9 of the Innovation Panel (2016), both of which are available fromthe UK Data Archive. Other analyses have explored initial response rates to this study, noting that just 16.5% of the invited sample completed the registration process and fewer still downloaded the app. In this study, the investigation of data quality involved defining and examining four measures of adherence to protocol, and the extent to which these aspects of adherence were sustained over the duration of the study period. The research identifies a reasonable level of engagement from those who agreed to participate in the app study. For example, the mean number of app use days in the one-month period was 21.7 and the mean number of spending events reported was 27.6. Almost all participants (96.6%) reported at least one spending event and of those, most (95%) used a combination of photographing receipts and making direct entries, or only photographed receipts, with 61% of all spending events reported by photographing receipts. Almost all of those (94.9%) who photographed one or more receipts which had relevant date information did so within, on average, 24 hours of the time of the spending event. Although adherence based on all four measures clearly declines across the study month, it remains reasonably high. This study provides encouragement for further development of the app, and further methodological research and experimentation to increase full and sustained adherence to protocol. If a spending study app is to be embedded successfully in a large-scale study such as Understanding Society, future efforts will inevitably focus on ways to raise initial participation rates, but it would be unfortunate if the particular benefits of app-based research, such as capturing detailed spending data from receipts using photographs, were entirely let go in favour of achieving higher initial response rates
Participation in a Mobile App Survey to Collect Expenditure Data as Part of a Large-Scale Probability Household Panel: Coverage and Participation Rates and Biases
Get PDFThis paper examines non-response in a mobile app study designed to collect expenditure data. We invited 2,383 members of the nationally representative Understanding Society Innovation Panel in Great Britain to download an app to record their spending on goods and services: participants were asked to scan receipts or report spending directly in the app every day for a month. We examine coverage of mobile devices and participation in the app study at different stages of the process. We use data from the prior wave of the panel to examine the prevalence of potential barriers to participation, including access, ability and willingness to use different mobile technologies. We also examine bias in who has devices and in who participates, considering socio-demographic characteristics, financial position and financial behaviours. While the participation rate was low, drop out was also low: over 80% of participants remained in the study for the full month. The main barriers to participation were access to, and frequency of use of mobile devices, willingness to download an app for a survey, and general cooperativeness with the survey. We found extensive coverage bias in who has and does not have mobile devices, and some bias in who participates conditional on having a device. In the full sample, biases remain in who participates in terms of socio-demographic characteristics and financial behaviours. Crucially, however, we observe no biases for several key correlates of spending
How could mobile devices improve survey measurement and social research? Exploring the opportunities and challenges through innovative examples of mobile data collection
No full textLessof Carli's presentation is about using mobile devices to improve survey measurement and social researc
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Get PDF“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Get PDFWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
Dispelling the Myths Behind First-author Citation Counts
Get PDFWe conducted a full-scale evaluative citation analysis study of scholars in the XML research field to explore just how different from each other author rankings resulting from different citation counting methods actually are, and to demonstrate the capability of emerging data and tools on the Web in supporting more realistic citation counting methods. Our results contest some common arguments for the continued
use of first-author citation counts in the evaluation of scholars, such as high correlations between author rankings by first-author citation counts and other citation
counting methods, and high costs of using more realistic citation counting methods that are not well-supported by the ISI databases. It is argued that increasingly available digital full text research papers make it possible for citation analysis studies to go beyond what the ISI databases have directly supported and to employ more
sophisticated methods
- …
