55 research outputs found
ETL: From the German Health Data Lab data formats to the OMOP Common Data Model
No full textObjective The German Health Data Lab is going to provide access to German statutory health insurance claims data ranging from 2009 to the present for research purposes. Due to evolving data formats within the German Health Data Lab, there is a need to standardize this data into a Common Data Model to facilitate collaborative health research and minimize the need for researchers to adapt to multiple data formats. For this purpose we selected transforming the data to the Observational Medical Outcomes Partnership Common Data Model. Methods We developed an Extract, Transform, and Load (ETL) pipeline for two distinct German Health Data Lab data formats: Format 1 (2009-2016) and Format 3 (2019 onwards). Due to the identical format structure of Format 1 and Format 2 (2017 -2018), the ETL pipeline of Format 1 can be applied on Format 2 as well. Our ETL process, supported by Observational Health Data Sciences and Informatics tools, includes specification development, SQL skeleton creation, and concept mapping. We detail the process characteristics and present a quality assessment that includes field coverage and concept mapping accuracy using example data. Results For Format 1, we achieved a field coverage of 92.7%. The Data Quality Dashboard showed 100.0% conformance and 80.6% completeness, although plausibility checks were disabled. The mapping coverage for the Condition domain was low at 18.3% due to invalid codes and missing mappings in the provided example data. For Format 3, the field coverage was 86.2%, with Data Quality Dashboard reporting 99.3% conformance and 75.9% completeness. The Procedure domain had very low mapping coverage (2.2%) due to the use of mocked data and unmapped local concepts The Condition domain results with 99.8% of unique codes mapped. The absence of real data limits the comprehensive assessment of quality. Conclusion The ETL process effectively transforms the data with high field coverage and conformance. It simplifies data utilization for German Health Data Lab users and enhances the use of OHDSI analysis tools. This initiative represents a significant step towards facilitating cross-border research in Europe by providing publicly available, standardized ETL processes (https://github.com/FraunhoferMEVIS/ETLfromHDLtoOMOP) and evaluations of their performance.</p
Real-world Evidence on Baseline Characteristics and Treatment in Metastatic Hormone-sensitive Prostate Cancer : Findings from the PIONEER 2.0 Big Data Investigation Group
No full textPIONEER Consortium collaborator list: Laurent Antoni, Charles Auffray, Anssi Auvinen, Chris Bangma, Anders Bjartell, Gabi Bernstein, Angelika Borkowetz, Danny Burke, Michael Bussmann, John Butler, Riccardo Campi, Simona Caputova, Laurence Colette, Louise Fullwood, Ronald Herrera, Thomas Hofmarcher, Marc Holtorf, Denis Horgan, Henkjan Huisman, Tim Hulsen, Andreas Josefsson, Daniel Kotik, Mark Lambrecht, Doron Lancet, Michael Lardas, Ailbhe Lawlor, Stephane Lejeune, Sophia Le Mare, Muriel Licour, Peter Lindgren, Elaine Longden-Chapman, Monika Maass, Maxim Moinat, Lisa Moris, Nicolas Mottet, Teemu Murtola, Kishore Papineni, Sarah Payne, Christian Reich, Kristin Reiche, Maria J. Ribal, Paul Robinson, Monique J. Roobol, Beth Russell, Vasileios Sakalis, Jack Schalken, Sarah Seager, Robert Shepherd, Aino Siltari, Emma Jane Smith, Azadeh Tafreshiha, Kirsi Talala, Teuvo Tammela, Derya Tilki, Patrizia Torremante, Sheela Tripathee, Kees van Bochove, Mieke Van Hemelrijck, Tapio Visakorpi, Marc Dietrich Voss, Jihong Zong, and Nazanin Zounemat Kermani.Peer reviewe
RADAR-base/radar-android-faros: Release 0.1.1
No full text<p>Changes from 0.1.0</p>
<ul>
<li>Updates to radar-commons-android 0.7.0</li>
<li>Improvements on artifact publishing</li>
</ul>
RADAR-base/Restructure-HDFS-topic: radar-hdfs-restructure version 0.5.2
No full text<p>Changes since version 0.5.1:</p>
<ul>
<li>Specify output file user and group</li>
<li>Added copyright statements in files added since version 0.5.x</li>
<li>Organised imports according to style guide.</li>
</ul>
RADAR-base/Restructure-HDFS-topic: radar-hdfs-restructure version 0.5.6
No full text<p>Changes since version 0.5.5:</p>
<ul>
<li>Corrects snappy decompression (fixes #43)</li>
</ul>
EHDEN - D4.6 - Final version of the Framework for quality benchmarking
No full textThe Data Quality Dashboard (DQD) developed as part of EHDEN Work Package 4 provides a comprehensive, customizable, and transparent way to both evaluate and communicate the quality of an OMOP CDM instance. It provides the code to run data quality checks against an OMOP CDM instance, and a way to visualize the results in a web application. The DQD is described in deliverable 4.2; this deliverable provides an update on progress and details of what was achieved in year two.
Each Data Partner runs the Data Quality Dashboard on the data at their site once it is converted to the OMOP CDM. Initially, it has been used to assess whether OMOP Standards such as primary key constraints and concept domain restrictions are being followed. Thinking specifically of the EHDEN federated network, providing an interactive data quality report for each participating site provides evidence not only that OMOP specifications were followed correctly but also demonstrates that the necessary due diligence was performed to ensure that the data are of research quality.
As the EHDEN network continues to grow and more Data Partners are at the point where they can run the DQD tool on their data we have been able to use it to assess adherence to CDM standards and readiness for network research. Such practical application will lead to continued innovation as we have already seen during year two. During the recent Rapid Collaboration Call for COVID-19 data sources, the DQD has been invaluable, providing insight into issues during the extract, transform, and load process and even serving as an education tool to disseminate standards and expectations.
The DQD has already been widely adopted across multiple observational health networks in addition to EHDEN. The National COVID Cohort Collaborative1, funded by the US National Institutes of Health, is building a comprehensive database of COVID patients from across the US. They employ the tool during their data ingestion and harmonization process to assess data as it is received. The US FDA Biologics Effectiveness and Safety System2 uses the DQD to evaluate data quality prior to engaging in observational research. Similarly, the OHDSI community has begun to require Data Partners to run the tool prior to participation in network research. In response to this support and usage a publication is in production to fully describe the tool and to increase dissemination. The DQD has proven to be a strong foundation for data quality reporting and a key tool to ensure confidence in the evidence generated through the EHDEN federated network and beyond
RADAR-base/radar-output-restructure: radar-output-restructure version 1.1.3
No full textChanges since version 1.1.2:
Updated Minio, including updated syntax
Read object tags from S3 and object metadata in Azure to determine end offset of file.
Wrapped JedisPool in a RedisHolder that does converts thrown JedisException to IOException.
Wrapped RadarKafkaRestructure and SourceDataCleaner into a Job to manage starting and catching exceptions. This fixes the occurrence that an OutOfMemoryException was not properly handled.
Fixed off-by-one error in OffsetIntervals
Reduced deduplication memory consumption, especially if the format: deduplication: distinctFields property is set
RADAR-base/radar-android-weather: radar-android-weather version 0.2.0
No full text<p>Changes since version 0.1:</p>
<ul>
<li>Update dependencies</li>
<li>List location requirement</li>
<li>Separated dynamic Weather API interface from static result</li>
</ul>
RADAR-base/radar-android-faros: radar-android-faros version 0.1.2
No full text<p>Changes since version 0.1.1:</p>
<ul>
<li>Additional logging</li>
<li>Use latest SDK (0.1.0)</li>
<li>Fixed README</li>
</ul>
- …
