1,720,986 research outputs found
Replication Data for: "Trustworthy Digital Repository Certification: A Longitudinal Study"
No full textThis dataset is a database compiled from two sources. First, we acquired all Data Seal of Approval (DSA) and early Core Trust Seal (CTS) audit reports from a MySQL database archived and made publicly accessible in DANS EASY (doi:10.17026/dans-28z-njxq). Second, we acquired more recent publicly accessible CTS audit reports from the list of certified repositories on the CTS website (https://www.coretrustseal.org/why-certification/certified-repositories/). We migrated and extracted the audit reports and their metadata from both sources into a file-based SQLite database that we have made publicly available on Harvard Dataverse. Our database allows you to replicate the tables and figures presented in the article, "Trustworthy Digital Repository Certification: A Longitudinal Study" that is published in Springer's indexed LNCS series for iConference 2023 (Donaldson, D.R., Russell, S.V. (2023). Trustworthy Digital Repository Certification: A Longitudinal Study. In I. Sserwanga, et al. (Eds.), Information for a Better World: Normality, Virtuality, Physicality, Inclusivity. iConference 2023. Lecture Notes in Computer Science, vol 13972. Springer, Cham. https://doi.org/10.1007/978-3-031-28032-0_42).
Our database is the entire set of self-assessment audit documents from the DSA and CTS certification programs as of October 2020. All the documents in the database follow the same format of a numerical score and narrative description of a repository’s compliance with each of the 16 guidelines. Although both have 16 scored sections, the guidelines for the earlier DSA and more recent CTS certification programs differ in the thematic arrangement of subtopics per section. Our database includes the section-by-section text and numerical scores of each repository’s audit reports, along with information used to identify repositories
The Perceived Value of Acquiring Data Seals of Approval Study Dataset and Associated Files
No full textThis package contains the data and associated analyses for a study which examined the benefits of acquiring Data Seals of Approval (DSAs) from the point of view of those who have them. This package includes data from a series of 15 semi-structured interviews with representatives from 16 different organizations in which participants described the benefits of having DSAs in their own words. The four files in this package include: 1) the coded interview transcripts and description of codes (i.e., codebook) in NVivo for Mac Version 11.3.2 (1888) file format (e.g., .nvpx), 2) the raw dataset that lists the frequency with which each benefit was mentioned by DSA board members and non-DSA board members in IBM SPSS Statistics 24 file format (e.g., .spv), and 3) the processed/analysed data from the Mann-Whitney U tests in two different file formats (e.g., .doc and .spv).This project was funded by a Research Data Alliance United States Data Share Fellowship from the Alfred P. Sloan Foundation.This study investigates the value of acquiring DSAs from the perspectives of those who have them. To avoid data collector characteristics and bias that could potentially affect what data were collected and how they were analysed, only those without prior experience in the development of the DSA were selected to be a part of the research team that handled data collection and analysis. Specifically, this study included a data collector who is knowledgeable about repository standards but yet did not play an active role in the development of any of them. The data collector also does not serve as a formal third-party auditor, a contrast from existing research on this topic. Approval for conducting this research was received from the Indiana University Human Subjects Office.
Our findings are drawn from data collected during interviews conducted between August 2015 and February 2016. All participants were at organizations whose repositories successfully acquired DSAs. We selected these individuals because only digital repository staff members at institutions which successfully acquired DSAs would be able to speak from experience about the actual benefits of having them. The list of 64 acquired seals on the DSA website constituted the sampling frame. The first author recruited participants by emailing representatives from each of the repositories that acquired the DSA and inviting their participation. The first author sent follow-up emails on two separate occasions to try to increase participation. As a result of these efforts, we successfully recruited 15 representatives from these repositories to participate in this study, with a response rate of 23%.
The primary purpose of conducting the semi-structured, 30-minute interviews was to understand the value of the audit process and certification from the perspective of actual digital repository staff members. The first author asked respondents to discuss: how they learned about the DSA certification, how they decided to undergo audit, how they prepared for it, what the process was like, any lessons learned, the perceived value of the audit process, and the perceived value of certification since attaining it. No incentives for participation were provided.
All interviews took place by telephone or via Skype and were audio recorded. Afterwards, all interviews were transcribed. Transcripts were then coded using NVivo – a qualitative data analysis software tool. Prior to analyzing the transcripts, the first author developed a codebook based primarily on the list of benefits on the DSA website. The first author also remained open to identifying additional themes as a result of analyzing the transcripts. The first author and a hired graduate student coded the transcripts. We calculated inter-rater reliability using Cohen’s Kappa. We achieved a score of 0.87; thus, on average, we agreed on codes 87% of the time.
Some participants played an active role in the development of the DSA, either as past or present DSA board members. This was seen as a potential threat to the validity of the data. In particular, DSA board members could subconsciously or consciously over-report the benefits of acquiring DSAs based on their knowledge of and experience with the standard. For this reason, additional data analyses were performed. Specifically, a series of Mann-Whitney U tests were performed to detect whether any statistically significant differences existed between DSA board members and non-DSA board members regarding the frequency with which they reported benefits of acquiring DSAs. All statistical analyses were performed using IBM SPSS Statistics 24.To access the coded interview transcripts and description of codes (i.e., codebook), you need NVivo for Mac Version 11.3.2 (1888). To access the raw dataset that lists the frequency with which each benefit was mentioned by DSA board members and non-DSA board members, you need IBM SPSS Statistics 24. To access the processed/analysed data from the Mann-Whitney U tests, you need IBM SPSS Statistics 24 to access the .spv file and Microsoft Word to access the .doc file
Provenance, End-User Trust and Reuse: An Empirical Investigation
Full text linkProvenance theorists and practitioners assume that provenance is essential for trust in and reuse of data. However, little empirical research has been conducted to more closely examine this assumption. This qualitative study explores how provenance affects end-users’ trust in and reuse of data. Toward this end, the authors conducted semi- structured interviews with 17 proteomics researchers who interact with data from ProteomeCommons.org, a large scientific data repository. Empirical findings from this study suggest that provenance does help end-users gauge the trustworthiness of data and build their confidence in reusing data. However, provenance also needs to be accompanied by other kinds of information, including: more specific data quality information, the data itself, and author reputation information. Implications of this study stress the value of end-user studies in provenance research, specifically to assess the ‘real-world’ impact of provenance encoded and communicated to end-users in systems.This material is based upon work supported by the National Science Founda- tion under Grant No. 090362
Development of a Scale for Measuring Perceptions of Trustworthiness for Digitized Archival Documents.
Full text linkTrustworthiness is the most fundamental but least well understood property
of digital repositories that hold and preserve archival documents. As these digital repositories scale in size and complexity, they are becoming essential sources for increasingly diverse populations of users. Scholarship across multiple disciplines has demonstrated that the trustworthiness of a digital repository tends to originate with organizational branding, surrounds and envelops the “control zone” of the managed digital space, and so resides primarily at the collective level of the repository. In spite of its conceptual centrality, little research has investigated trustworthiness of the documentary contents of repositories as conceived by the designated communities of users that the repository is intended to serve. This dissertation investigates users’ perceptions of trustworthiness for archival documents housed in a large, heterogeneous, government‐run digital repository. This dissertation utilizes the methodology of scale development, which involves four steps: 1) Construct Definition, 2) Generating an Item Pool, 3)Designing the Scale, and 4) Full Administration and Item Analysis. To address Steps 1 and 2 of scale development, I conducted a focus group study to elicit perspectives
on trustworthiness and identify items for measurement of trustworthiness based
upon actual users’ articulation of the concept; twenty‐two genealogists who
regularly utilize documents preserved by the Washington State Digital Archives participated. To address Steps 3 and 4 of scale development, I conducted
quantitative survey research and evaluated the responses of 233 genealogists,
including constructing and testing an original Digitized Archival Document
Trustworthiness Scale (DADTS). I also validated DADTS with a sample of users
beyond the participants who were used to develop it. DADTS specifies the
components of trustworthiness and also demonstrates the measurability of the
concept within a digital repository context at the document level.
This dissertation advances scholarship on trustworthiness in three ways.
First, it revises an existing conceptual model for trustworthiness perception.
Second, it creates an original measurement model for digitized archival document
trustworthiness perception—the Digitized Archival Document Trustworthiness
Scale (DADTS). Third, it contributes to a deeper understanding of the concept of
trustworthiness by providing measurement of the concept in a way that is sensitive to its nuances.PhDInformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/111489/1/devand_1.pd
The Digitized Archival Document Trustworthiness Scale Study Dataset and Associated Files
No full textThis package contains the data and associated analyses for a study which built, tested, and assessed a scale for measuring genealogists' perceptions of trustworthiness for digitized genealogical records, including digitized marriage, death, and birth records. The study participants were genealogists who frequently utilize documents housed in the Washington State Digital Archives (WADA).The four files in this package include: 1) the dataset [size = 393 KB] that was used to perform the statistical analyses in SPSS 23.0, 2) the output [size = 1.3 MB] of the analysis in Microsoft Word format, 3) the output [size = 49 KB] of the analysis in SPSS format, and 4) the syntax file [size = 910 Bytes] that can be used to replicate the analyses in SPSS. The dataset is the raw/unanalyzed data, both output files are the same processed/analyzed data in two different file formats, and the syntax file lists the code that was used to perform the analyses.Scale development was used to collect and analyze the data.Scale development involves four primary steps (DeVellis, 2012; Spector, 1992): Step 1—Construct Definition, Step 2—Generating an Item Pool, Step 3—Designing the Scale, and Step 4—Full Administration and Item Analysis. Step 1 of scale development is to construct a definition. In this study, it involved a review of the literature to identify the scope of trustworthiness for the purpose of empirical investigation. Step 1 also involved focus groups to understand how members of a designated community (i.e., genealogists) talk about trustworthiness. The findings from the focus groups are reported elsewhere (Donaldson & Conway, 2015). Step 2 of scale development is to generate an item pool. It involved identifying items for measuring trustworthiness from multiple sources, including the literature, subject matter experts, and focus groups data (Donaldson & Conway, 2015). Step 3 of scale development is to design the scale. It involved transforming the item pool resulting from Step 2 into a web survey for pretesting and refinement. Step 4 of scale development is the full administration of the survey and subsequent item analysis. It involved administering the final item pool comprising items gathered from earlier steps of scale development to a large sample of designated community members for their evaluation. Each item described a circumstance one might encounter while using a digitized archival document. Participants answered whether the circumstance described by each item would cause them to perceive a digitized archival document as either untrustworthy or trustworthy on a 7‐point scale: very untrustworthy, untrustworthy, slightly untrustworthy, neither untrustworthy nor trustworthy, slightly trustworthy, trustworthy, or very trustworthy. An eighth option, “Not Applicable,” was included for participants to choose if the circumstance an item described was not relevant to their experience of using digitized archival documents. Step 4 also involved analyzing designated community members’ responses via factor analysis to identify the items that were most essential for measuring the trustworthiness of preserved information (in this case, digitized genealogical records). Two types of analysis were performed: item analysis and exploratory factor analysis. Item analysis involved analysis of item variances, item‐total correlations, item means, and item standard deviations (DeVellis, 2012). To assess item variances, the range of responses (i.e., items’ minimums and maximums) for each item were inspected. To assess item-total correlations, each item was examined to determine the extent to which it correlated with the collection of remaining items. Items’ means and standard deviations were examined to ensure that, for each item, the means were near the midpoint of the 7-point scale on which participants rated the items while also ensuring that there was variation involved in attaining the means. Standard deviations above zero indicated that not everyone provided the same ratings for each item to arrive at a mean, but instead, different participants provided different ratings of each item to arrive at a mean.
After performing item analysis, exploratory factor analysis (EFA) was conducted using SPSS Statistics 22.0, a software package for statistical analysis, to establish the factor structure of the trustworthiness items (Kline, 2013). EFA was used as a tool to help identify the most critical items for measurement of trustworthiness. “Important” trustworthiness items were operationalized as items with high factor loadings on factors with large eigenvalues. To assign items to factors, factor loadings equal to or higher than .32 were considered (Tabachnick & Fidell, 2001). Two tests were performed to assess the appropriateness of the data that were collected during this study for EFA: the Kaiser‐Meyer‐Olkin Measure of Sampling Adequacy (Kaiser, 1970; Kaiser & Rice, 1974) and Bartlett’s Test of Sphericity (Bartlett, 1954). Afterwards, EFA was conducted using principal axis factoring with oblique rotation; oblique rotation allows factors to correlate (Kline, 2013). Since the items were all trustworthiness items, there was no reason to think that any factors underlying the items would not be correlated, hence the decision to employ oblique rotation. Results of Cattell’s (1966) scree test were used to determine the number of factors to retain.SPSS 22.0 and SPSS 23.0 were used to access the SPSS files (e.g., the .sav, .sps, .spv files). Microsoft Word 2011 for Mac was used to access the .doc file
Users’ Trust in Trusted Digital Repository Content: Paper - iPRES 2011 - Singapore
No full textScholars who study trust in digital archives have largely focused their attention on the power of certification by third-party audit as a way to communicate trustworthiness to end-users. In doing so, they assume that the establishment of a network of trusted digital archives will create a climate of trust. But certification at the repository level also assumes the trustworthiness of digital objects within a repository; specifically that digital repository objects are authentic and reliable. This paper proposes the use of document-level seals of approval as a means of communicating to end-users about the trustworthiness of digital objects that is commensurate with specific user interaction. Implications of this proposed research stress the importance of assessing the ‘real-world’ impact of trust signals on users
The Digitized Archival Document Trustworthiness Scale
Full text linkDesignated communities are central to validation of preservation. If a designated community is able to understand and use information found within a digital repository, the assumption is that the information has been properly preserved. As judging the trustworthiness of information requires at least some level of understanding of that information, this paper presents results of a study aimed at developing a tool for measuring designated community members’ perceptions of trustworthiness for preserved information found within a digital repository. The study focuses on genealogists at the Washington State Digital Archives who routinely interact with digitized genealogical records, including digitized marriage, death, and birth records. Results of the study include construction of an original Digitized Archival Document Trustworthiness Scale (DADTS). DADTS is a ready-made tool for digital curators to use to measure the trustworthiness perceptions of their designated community members. Implications of this study include the feasibility of engaging members of a designated community in the construction of a scale for measuring trustworthiness perception, thereby providing deeper insight into the understandability and usability of preserved information by that designated community.Â
Users’ Trust in Trusted Digital Repository Content
Full text linkScholars who study trust in digital archives have largely focused their attention on the power of certification by third-party audit as a way to communicate trustworthiness to end-users. In doing so, they assume that the establishment of a network of trusted digital archives will create a climate of trust. But certification at the repository level also assumes the trustworthiness of digital objects within a repository; specifically that digital repository objects are authentic and reliable. This paper proposes the use of document-level seals of approval as a means of communicating to end-users about the trustworthiness of digital objects that is commensurate with specific user interaction. Implications of this proposed research stress the importance of assessing the ‘real-world’ impact of trust signals on users
Measuring Perceptions of Trustworthiness: A Research Project
Full text linkThe digital curation and preservation community has long acknowledged that trustworthiness is a critical component of successful digital repositories. However, there is no known method to determine if or under what circumstances an end-user perceives a repository as trustworthy. While the research literature describes definitions, criteria, and certification processes that allow repository managers to assert trustworthiness under certain conditions, it does not adequately define, measure, or specify trustworthiness from the perspective of the end-user. This paper highlights traditional notions of trustworthiness in the context of the literature on digital repositories and explores trustworthiness from the end-user’s perspective. The paper also presents an ongoing research project to: (1) investigate designated communities’ perspectives on trustworthiness using focus groups, and (2) explore building, testing, and assessing an index to measure trustworthiness.Horace H. Rackham School of Graduate Studies, University of Michigan, Ann Arbo
Trust in Archives-Trust in Digital Archival Content Framework
No full textThis record is for a(n) offprint of an article published in Archivaria The Journal of the Association of Canadian Archivists on 2019-11-14.Trust in information found online matters now more than ever. The proliferation of fake news and “post-truths” makes it difficult for people to determine what is trustworthy. People even question digital versions of basic identity documents such as birth certificates, the processes that archivists use to preserve them, and attestations of their authenticity. Given the current landscape, we need to better understand trust in archival materials found online. Drawing from relevant literature, I developed a model to examine the relationship between trust in archives and trust in digital archival content, the Trust in Archives–Trust in Digital Archival Content (TIA-TDAC) Framework. This article outlines an empirical test of the TIA-TDAC Framework. Using measures of both levels of trust, I designed a survey to evaluate archives users and potential archives users’ trust in a diverse range of digitized and born-digital materials from a broad range of archives in the United States. Results of the survey indicate both that the association of trust in archives with trust in digital archival content could be empirically measured and that the relationship between the two trust levels was positively correlated. The article also discusses future directions for research.offprin
- …
