7 research outputs found
Can GPT-3.5 Generate and Code Discharge Summaries?
No full textObjective: To investigate GPT-3.5 in generating and coding medical documents with ICD-10 codes for data augmentation on low-resources labels.
Materials and Methods: Employing GPT-3.5 we generated and coded 9,606 discharge summaries based on lists of ICD-10 code descriptions of patients with infrequent (generation) codes within the MIMIC-IV dataset. Combined with the baseline training set, this formed an augmented training set. Neural coding models were trained on baseline and augmented data and evaluated on a MIMIC-IV test set. We report micro- and macro-F1 scores on the full codeset, generation codes, and their families. Weak Hierarchical Confusion Matrices were employed to determine within-family and outside-of-family coding errors in the latter codesets. The coding performance of GPT-3.5 was evaluated both on prompt-guided self-generated data and real MIMIC-IV data. Clinical professionals evaluated the clinical acceptability of the generated documents.
Results: Augmentation slightly hinders the overall performance of the models but improves performance for the generation candidate codes and their families, including one unseen in the baseline training data. Augmented models display lower out-of-family error rates. GPT-3.5 can identify ICD-10 codes by the prompted descriptions, but performs poorly on real data. Evaluators note the correctness of generated concepts while suffering in variety, supporting information, and narrative.
Discussion and Conclusion: GPT-3.5 alone is unsuitable for ICD-10 coding. Augmentation positively affects generation code families but mainly benefits codes with existing examples. Augmentation reduces out-of-family errors. Discharge summaries generated by GPT-3.5 state prompted concepts correctly but lack variety, and authenticity in narratives. They are unsuitable for clinical practice.15 pages; 250 words in abstract; 4,152 words in main body; 4 figures (1 black and white, 3 colour); 4 tables; 34 references; Accepted and published by the Journal of the American Medical Informatics Associatio
Prevalence and risk factors for long COVID among adults in Scotland using electronic health records : a national, retrospective, observational cohort study
No full textBackground: Long COVID is a debilitating multisystem condition. The objective of this study was to estimate the prevalence of long COVID in the adult population of Scotland, and to identify risk factors associated with its development. Methods: In this national, retrospective, observational cohort study, we analysed electronic health records (EHRs) for all adults (≥18 years) registered with a general medical practice and resident in Scotland between March 1, 2020, and October 26, 2022 (98–99% of the population). We linked data from primary care, secondary care, laboratory testing and prescribing. Four outcome measures were used to identify long COVID: clinical codes, free text in primary care records, free text on sick notes, and a novel operational definition. The operational definition was developed using Poisson regression to identify clinical encounters indicative of long COVID from a sample of negative and positive COVID-19 cases matched on time-varying propensity to test positive for SARS-CoV-2. Possible risk factors for long COVID were identified by stratifying descriptive statistics by long COVID status. Findings: Of 4,676,390 participants, 81,219 (1.7%) were identified as having long COVID. Clinical codes identified the fewest cases (n = 1,092, 0.02%), followed by free text (n = 8,368, 0.2%), sick notes (n = 14,469, 0.3%), and the operational definition (n = 64,193, 1.4%). There was limited overlap in cases identified by the measures; however, temporal trends and patient characteristics were consistent across measures. Compared with the general population, a higher proportion of people with long COVID were female (65.1% versus 50.4%), aged 38–67 (63.7% versus 48.9%), overweight or obese (45.7% versus 29.4%), had one or more comorbidities (52.7% versus 36.0%), were immunosuppressed (6.9% versus 3.2%), shielding (7.9% versus 3.4%), or hospitalised within 28 days of testing positive (8.8% versus 3.3%%), and had tested positive before Omicron became the dominant variant (44.9% versus 35.9%). The operational definition identified long COVID cases with combinations of clinical encounters (from four symptoms, six investigation types, and seven management strategies) recorded in EHRs within 4–26 weeks of a positive SARS-CoV-2 test. These combinations were significantly (p < 0.0001) more prevalent in positive COVID-19 patients than in matched negative controls. In a case-crossover analysis, 16.4% of those identified by the operational definition had similar healthcare patterns recorded before testing positive. Interpretation:The prevalence of long COVID presenting in general practice was estimated to be 0.02–1.7%, depending on the measure used. Due to challenges in diagnosing long COVID and inconsistent recording of information in EHRs, the true prevalence of long COVID is likely to be higher. The operational definition provided a novel approach but relied on a restricted set of symptoms and may misclassify individuals with pre-existing health conditions. Further research is needed to refine and validate this approach
Deriving and validating a risk prediction model for long COVID : a population-based, retrospective cohort study in Scotland
No full textObjectives Using electronic health records, we derived and internally validated a prediction model to estimate risk factors for long COVID and predict individual risk of developing long COVID. Design Population-based, retrospective cohort study. Setting Scotland. Participants Adults (≥18 years) with a positive COVID-19 test, registered with a general medical practice between 1 March 2020 and 20 October 2022. Main outcome measures Adjusted odds ratios (aORs) with 95% confidence intervals (CIs) for predictors of long COVID, and patients’ predicted probabilities of developing long COVID. Results A total of 68,486 (5.6%) patients were identified as having long COVID. Predictors of long COVID were increasing age (aOR: 3.84; 95% CI: 3.66–4.03 and aOR: 3.66; 95% CI: 3.27–4.09 in first and second splines), increasing body mass index (BMI) (aOR: 3.17; 95% CI: 2.78–3.61 and aOR: 3.09; 95% CI: 2.13–4.49 in first and second splines), severe COVID-19 (aOR: 1.78; 95% CI: 1.72–1.84); female sex (aOR: 1.56; 95% CI: 1.53–1.60), deprivation (most versus least deprived quintile, aOR: 1.40; 95% CI: 1.36–1.44), several existing health conditions. Predictors associated with reduced long COVID risk were testing positive while Delta or Omicron variants were dominant, relative to when the Wild-type variant was dominant (aOR: 0.85; 95% CI: 0.81–0.88 and aOR: 0.64; 95% CI: 0.61–0.67, respectively) having received one or two doses of COVID-19 vaccination, relative to unvaccinated (aOR: 0.90; 95% CI: 0.86–0.95 and aOR: 0.96; 95% CI: 0.93–1.00). Conclusions Older age, higher BMI, severe COVID-19 infection, female sex, deprivation and comorbidities were predictors of long COVID. Vaccination against COVID-19 and testing positive while Delta or Omicron variants were dominant predicted reduced risk
Deriving and validating a risk prediction model for long COVID: a population-based, retrospective cohort study in Scotland
No full textObjectivesUsing electronic health records, we derived and internally validated a prediction model to estimate risk factors for long COVID and predict individual risk of developing long COVID.DesignPopulation-based, retrospective cohort study.Setting ScotlandParticipantsAdults (≥18 years) with a positive COVID-19 test, registered with a general medical practice between March 1, 2020 and October 20, 2022.Main outcome measuresAdjusted odds ratios (aORs) with 95% confidence intervals (CIs) for predictors of long COVID, and patients’ predicted probabilities of developing long COVID.Results68,486 (5.6%) patients were identified as having long COVID. Predictors of long COVID were increasing age (aOR 3.84; 95%CI 3.66-4.03 and aOR 3.66 95%CI 3.27-4.09 in first and second splines), increasing body mass index (BMI) (aOR 3.17; 95%CI 2.78-3.61 and aOR 3.09 95%CI 2.13-4.49 in first and second splines), severe COVID-19 (aOR 1.78; 95%CI 1.72-1.84); female sex (aOR 1.56; 95%CI 1.53-1.60), deprivation (most versus least deprived quintile, aOR 1.40; 95%CI 1.36-1.44), several existing health conditions. Predictors associated with reduced long COVID risk were testing positive while Delta or Omicron variants were dominant, relative to when the Wild-type variant was dominant (aOR 0.85; 95%CI 0.81-0.88 and aOR 0.64; 95%CI 0.61-0.67, respectively) having received one or two doses of COVID-19 vaccination, relative to unvaccinated (aOR 0.90; 95%CI 0.86-0.95 and aOR 0.96; 95%CI 0.93-1.00). ConclusionsOlder age, higher BMI, severe COVID-19 infection, female sex, deprivation, and comorbidities were predictors of long COVID. Vaccination against COVID-19 and testing positive while Delta or Omicron variants were dominant predicted reduced risk
Prevalence and risk factors for long COVID among adults in Scotland using electronic health records : a national, retrospective, observational cohort study
No full textAcknowledgements This work was supported by the Chief Scientist Office, grant number COV/LTE/20/15. EAVE II is supported by a grant (MC_PC_19075) from the Medical Research Council; and a grant (MC_PC_19004) from BREATHE–The Health Data Research Hub for Respiratory Health, funded through the UK Research and Innovation Industrial Strategy Challenge Fund. LD was supported by a post-doctoral clinical fellowship from the Asthma UK Centre for Applied Research. SVK acknowledges funding from a NRS Senior Clinical Fellowship (SCAF/15/02), the Medical Research Council (MC_UU_00022/2) and the Scottish Government Chief Scientist Office (SPHSU17). The authors would like to acknowledge the support of Dave Kelly and Lamorna Brown of Albasoft Ltd., and Sharon Kennedy, Mike Birnie, Safraj Shahul Hameed and Elliott Hall of Public Health Scotland for their involvement in obtaining approvals, provisioning, and linking data and the use of the secure analytical platform within the National Safe Haven. Funding Chief Scientist Office (Scotland), Medical Research Council, and BREATHE.Peer reviewe
