238 research outputs found
Protocol for the development of the Chatbot Assessment Reporting Tool (CHART) for clinical advice
No full textIntroduction Large language model (LLM)-linked chatbots are being increasingly applied in healthcare due to their impressive functionality and public availability. Studies have assessed the ability of LLM-linked chatbots to provide accurate clinical advice. However, the methods applied in these Chatbot Assessment Studies are inconsistent due to the lack of reporting standards available, which obscures the interpretation of their study findings. This protocol outlines the development of the Chatbot Assessment Reporting Tool (CHART) reporting guideline.Methods and analysis The development of the CHART reporting guideline will consist of three phases, led by the Steering Committee. During phase one, the team will identify relevant reporting guidelines with artificial intelligence extensions that are published or in development by searching preprint servers, protocol databases, and the Enhancing the Quality and Transparency of health research Network. During phase two, we will conduct a scoping review to identify studies that have addressed the performance of LLM-linked chatbots in summarising evidence and providing clinical advice. The Steering Committee will identify methodology used in previous Chatbot Assessment Studies. Finally, the study team will use checklist items from prior reporting guidelines and findings from the scoping review to develop a draft reporting checklist. We will then perform a Delphi consensus and host two synchronous consensus meetings with an international, multidisciplinary group of stakeholders to refine reporting checklist items and develop a flow diagram.Ethics and dissemination We will publish the final CHART reporting guideline in peer-reviewed journals and will present findings at peer-reviewed meetings. Ethical approval was submitted to the Hamilton Integrated Research Ethics Board and deemed “not required” in accordance with the Tri-Council Policy Statement (TCPS2) for the development of the CHART reporting guideline (#17025)
Reporting guideline for chatbot health advice studies: the chatbot assessment reporting tool (CHART) statement
No full textThe Chatbot Assessment Reporting Tool (CHART) is a reporting guideline developed to provide reporting recommendations for studies evaluating the performance of generative artificial intelligence (AI)-driven chatbots when summarizing clinical evidence and providing health advice, referred to as chatbot health advice studies. CHART was developed in several phases after performing a comprehensive systematic review to identify variation in the conduct, reporting, and method in chatbot health advice studies. Findings from the review were used to develop a draft checklist that was revised through an international, multidisciplinary, modified, asynchronous Delphi consensus process of 531 stakeholders, three synchronous panel consensus meetings of 48 stakeholders, and subsequent pilot testing of the checklist. CHART includes 12 items and 39 subitems to promote transparent and comprehensive reporting of chatbot health advice studies. These include title (subitem 1a), abstract/summary (subitem 1b), background (subitems 2a,b), model identifiers (subitems 3a,b), model details (subitems 4a-c), prompt engineering (subitems 5a,b), query strategy (subitems 6a-d), performance evaluation (subitems 7a,b), sample size (subitem 8), data analysis subitem 9a), results (subitems 10a-c), discussion (subitems 11a-c), disclosures (subitem 12a), funding (subitem 12b), ethics (subitem 12c), protocol (subitem 12d), and data availability (subitem 12e). The CHART checklist and corresponding diagram of the method were designed to support key stakeholders including clinicians, researchers, editors, peer reviewers, and readers in reporting, understanding, and interpreting the findings of chatbot health advice studies
Protocol for the development of the Chatbot Assessment Reporting Tool (CHART) for clinical advice
No full textIntroduction: Large language model (LLM)-linked chatbots are being increasingly applied in healthcare due to their impressive functionality and public availability. Studies have assessed the ability of LLM-linked chatbots to provide accurate clinical advice. However, the methods applied in these Chatbot Assessment Studies are inconsistent due to the lack of reporting standards available, which obscures the interpretation of their study findings. This protocol outlines the development of the Chatbot Assessment Reporting Tool (CHART) reporting guideline.
Methods and analysis: The development of the CHART reporting guideline will consist of three phases, led by the Steering Committee. During phase one, the team will identify relevant reporting guidelines with artificial intelligence extensions that are published or in development by searching preprint servers, protocol databases, and the Enhancing the Quality and Transparency of health research Network. During phase two, we will conduct a scoping review to identify studies that have addressed the performance of LLM-linked chatbots in summarising evidence and providing clinical advice. The Steering Committee will identify methodology used in previous Chatbot Assessment Studies. Finally, the study team will use checklist items from prior reporting guidelines and findings from the scoping review to develop a draft reporting checklist. We will then perform a Delphi consensus and host two synchronous consensus meetings with an international, multidisciplinary group of stakeholders to refine reporting checklist items and develop a flow diagram.
Ethics and dissemination: We will publish the final CHART reporting guideline in peer-reviewed journals and will present findings at peer-reviewed meetings. Ethical approval was submitted to the Hamilton Integrated Research Ethics Board and deemed “not required” in accordance with the Tri-Council Policy Statement (TCPS2) for the development of the CHART reporting guideline (#17025).
Registration: This study protocol is preregistered with Open Science Framework: https://doi.org/10.17605/OSF.IO/59E2Q
Fidelity monitoring across the seven studies in the Consortium of Hospitals Advancing Research on Tobacco (CHART)
No full textBackground
This paper describes fidelity monitoring (treatment differentiation, training, delivery, receipt and enactment) across the seven National Institutes of Health-supported Consortium of Hospitals Advancing Research on Tobacco (CHART) studies. The objectives of the study were to describe approaches to monitoring fidelity including treatment differentiation (lack of crossover), provider training, provider delivery of treatment, patient receipt of treatment, and patient enactment (behavior) and provide examples of application of these principles.
Methods
Conducted between 2010 and 2014 and collectively enrolling over 9500 inpatient cigarette smokers, the CHART studies tested different smoking cessation interventions (counseling, medications, and follow-up calls) shown to be efficacious in Cochrane Collaborative Reviews. The CHART studies compared their unique treatment arm(s) to usual care, used common core measures at baseline and 6-month follow-up, but varied in their approaches to monitoring the fidelity with which the interventions were implemented.
Results
Treatment differentiation strategies included the use of a quasi-experimental design and monitoring of both the intervention and control group. Almost all of the studies had extensive training for personnel and used a checklist to monitor the intervention components, but the items on these checklists varied widely and were based on unique aspects of the interventions, US Public Health Service and Joint Commission smoking cessation standards, or counselor rapport. Delivery of medications ranged from 31 to 100 % across the studies, with higher levels from studies that gave away free medications and lower levels from studies that sought to obtain prescriptions for the patient in real world systems. Treatment delivery was highest among those studies that used automated (interactive voice response and website) systems, but this did not automatically translate into treatment receipt and enactment. Some studies measured treatment enactment in two ways (e.g., counselor or automated system report versus patient report) showing concurrence or discordance between the two measures.
Conclusions
While fidelity monitoring can be challenging especially in dissemination trials, the seven CHART studies used a variety of methods to enhance fidelity with consideration for feasibility and sustainability.
Trial registration
- Dissemination of Tobacco Tactics for hospitalized smokers. Clinical Trials Registration No. NCT01309217.
- Smoking cessation in hospitalized smokers. Clinical Trials Registration No. NCT01289275.
- Using “warm handoffs” to link hospitalized smokers with tobacco treatment after discharge: study protocol of a randomized controlled trial. Clinical Trials Registration No. NCT01305928.
- Web-based smoking cessation intervention that transitions from inpatient to outpatient. Clinical Trials Registration No. NCT01277250.
- Effectiveness of smoking-cessation interventions for urban hospital patients. Clinical Trials Registration No. NCT01363245.
- Comparative effectiveness of post-discharge interventions for hospitalized smokers. Clinical Trials Registration No. NCT01177176.
- Health and economic effects from linking bedside and outpatient tobacco cessation services for hospitalized smokers in two large hospitals. Clinical Trials Registration No. NCT01236079
The arabin pessary to prevent preterm birth in women with a twin pregnancy and a short cervix : The STOPPIT 2 RCT
No full textFunding Information: The research reported in this issue of the journal was funded by the HTA programme as project number 13/04/22. The contractual start date was in November 2014. The draft report began editorial review in September 2020 and was accepted for publication in February 2021. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report. Declared competing interests of authors: Jane E Norman has received grants from government and charitable bodies for research into understanding the mechanism of term and preterm labour and understanding treatments. Within the last 3 years, Jane E Norman has acted on a Data Safety and Monitoring Board for a study involving a preterm birth therapeutic agent for GlaxoSmithKline plc (GlaxoSmithKline plc, Brentford, UK) and has provided consultancy for a small pharmaceutical company (Dilafor AB, Solna, Sweden) on drugs to alter labour progress. She was on the Health Technology Assessment (HTA) Maternal Neonatal and Child Health Panel (2013–18) and she was a member of the National Institute for Health Research (NIHR) HTA and Efficacy and Mechanism Evaluation (EME) Editorial Board (2012–14). John Norrie reports grants from the University of Edinburgh (Edinburgh, UK) during the conduct of the study, and declares that he is or has been a member of the following: HTA Commissioning Sub-Board (EOI) (2012–16), NIHR CTU Standing Advisory Committee (2017–present), NIHR HTA and EME Editorial Board (2014–19), Pre-Exposure Prophylaxis Impact Review Panel (2017–present), EME Strategy Advisory Committee (2019–present), EME – Funding Committee Members (2019–present), EME Funding Committee Sub-Group Remit & Comp Check (2019–present), HTA General Committee (2016–19), HTA Funding Committee Policy Group (formerly Clinical Studies Group) (2016–19), HTA Commissioning Committee (2010–16) and was a member of the HTA and EME Editorial Board between 2014 and 2019. Sarah Cunningham-Burley reports personal fees and other from the Wellcome Trust (London, UK), other from the University of Copenhagen (Copenhagen, Denmark), other funding from NIHR Global Health Research, personal fees from the French National Cancer Institute (Paris, France) and personal fees from the Health Research Board (Dublin, Ireland), outside the submitted work. Andrew Shennan is a member of the NIHR HTA Commissioning Committee (2018–22). Stephen C Robson was a member of the NIHR EME Funding Committee (2012–15). Steven Thornton is a trustee of a number of charities, including those that fund related research. He reports personal fees from GlaxoSmithKline plc, during the conduct of the study and outside the submitted work, and personal fees from Johnson & Johnson (Johnson & Johnson, Brunswick, NJ, USA) for consulting services. He holds positions in the Royal College of Obstetricians and Gynaecologists (London, UK) and other organisations. He was a member of the NIHR EME Strategy Advisory Committee (2018–19), EME – Funding Committee Members (2015–19), EME Funding Committee Sub-Group Remit & Comp Check (2018–19) and the Medical Research Council Multimorbidity Board (2020). Neil Marlow reports personal fees from Shire-Takeda (London, UK), Novartis Pharmaceuticals UK Ltd (London, UK) and GlaxoSmithKlein plc, outside the submitted work. Sarah J Stock declares that she is a member of the NIHR HTA General Committee (2016–22). In addition, Sarah J Stock received other research funding from the NIHR (14/32/01 QUIDS), Wellcome Trust (209560/Z/17/Z) and Chief Scientist Office (Edinburgh, UK), during the course of the study. Philip R Bennett reports personal fees and membership of a scientific panel from ObsEva (Plan-les-Ouates, Switzerland), outside the submitted work. In addition, Philip R Bennett has a patent PCT/GB1997/000529 WO1997031631 A1 ‘COX-2 selective inhibitors for managing labour and uterine contractions’ issued, a patent PCT/GB2004/001380 WO2005053705 A1 ‘Use of a cyclopentenone prostaglandin for delaying the onset and/or preventing the continuation of labour’ (priority date 2 December 2003) issued, a patent PCT/GB2016/050618 ‘Circulating miRNAs predictive of cervical shortening and preterm birth’ (pending UK filing 6 March 2015/full international filing completed 7 March 2016) issued, a patent PCT/GB2016/ 050621 ‘Rapid evaporative ionisation mass spectroscopy (REIMS) and desorbtion electrospray ionisation mass spectroscopy (DESI-MS) analysis of swabs and biopsy samples’ (pending UK filing 6 March 2015/full international filing completed 7 March 2016) pending, a patent PCT/GB2019 ‘Desorbtion electrospray ionisation mass spectroscopy (DESI-MS) analysis of swabs to predict vaginal microbiota’ (pending UK filing March 2019) pending, and a patent PCT/GB2019/ ‘Circulating miRNAs predictive of IUGR’ (pending UK filing March 2019) pending
The EPICure study : growth and blood pressure at 6 years of age following extremely preterm birth
No full textBackground: Preterm children are at risk for reduced growth in early childhood, which may predispose them to later changes in blood pressure. We studied growth and blood pressure (BP) in extremely preterm (EP) children at age 6 years.
Methods: We evaluated children who were born at 25 completed weeks of gestation or less in the United Kingdom and Ireland in 1995 when they reached early school age. Children underwent standardized assessments, including auxology and sitting blood pressure.
Results: Of 308 surviving children, 241 (78 percent) were assessed at a median age of 6 years 4 months; 160 full term classmates acted as a comparison group. Compared to classmates, EP children were 1.2 standard deviations (SD) lighter, 0.97SD shorter, BMI was 0.95SD lower and head circumference 1.3SD lower. Compared to 2.5 years of age, EP children showed catch up in terms of weight by 0.37SD, height by 0.42SD and head circumference by 0.13SD. Systolic
and diastolic BP were lower by 2.3mmHg and 2.4mmHg respectively in EP children but these differences were accounted for by differences in height and BMI. Maternal smoking in pregnancy was associated with lower BP, children born before 24 weeks had higher systolic and children given postnatal steroids higher diastolic pressures.
Conclusions: Poor postnatal growth seen after birth and at in the third year persists into school age. Catch up growth reduces some of the early deficit but is least for head growth.
Despite serious postnatal growth restriction blood pressure appears similar in both EP and term classmates
Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups
No full textOBJECTIVE: To examine the efficacy of different methods of detecting a high death rate and determining whether an increase in deaths after heart transplantation could be explained by chance. DESIGN:Retrospective analysis of deaths after heart transplantation. Seven methods were used: mortality above national average, mortality excessively above national average, test of moving average mortality, test of number of consecutive deaths, sequential probability ratio test (SPRT), cusum graph with v-mask, and CRAM chart. The national average mortality was not available and a rate of 15% was used instead as the benchmark. SETTING: Regional cardiothoracic unit. PARTICIPANTS: All 371 patients who received a heart transplant in the programme, 1986-2000. MAIN OUTCOME MEASURES: 30 day survival after transplantation. RESULTS: All methods provided evidence that the 30 day mortality had been high at some stage. The probability that the finding was a false positive depended on which test was used. At the end of the series the average mortality, sequential probability ratio, and cusum tests indicated a level of deaths higher than the benchmark while the remaining four tests yielded negative results. CONCLUSIONS:If the decision to test for outlying mortality is made retrospectively, in the light of the data, it is not possible to determine the false positive rate. Prospective on-site mortality monitoring with the CRAM chart is recommended as this method can quantify the death rate and identify periods when an audit of cases is indicated, even when data from other institutions are not available. A hospital mortality monitoring group can routinely monitor all deaths in the hospital, by specialty, using hospital episode statistics (HES) data and appropriate statistical methods
Properties of the cumulative risk-adjusted mortality (CRAM) chart, including the number of deaths before a doubling of the death rate is detected.
No full textThe cumulative risk-adjusted mortality (CRAM) chart is intended to detect changes in the failure rate in a sequence of trials when the relative risk but not the absolute risk of failure at each trial is known. For example, surgical mortality can be monitored taking into account the heterogeneity of patients and procedures. We simulate abrupt change to twice or half the previous failure rate and series with no change. As with any control procedure, changes are eventually signaled whether or not real changes have occurred. With initial mean failure rates ranging from 1% to 25%, the median number of failures before a doubling is signaled is about 20 (i.e., about 10 extra failures) and is nearly independent of the extent of heterogeneity in the series. CRAM chart calculations give an estimate of the risk of failure at the next trial, and we document the performance of this estimate
Pengaruh Penerapan Ongoing Assesment dalam Pembelajaran CTL Menggunakan Flip Chart
No full textThis study aims to determine the effect of ongoing assesment in collaborative teamwork learning using flip chart on students’ collaborative abilities and cognitive skills. This research design used was one shot case design and one group pretest-posttest design. This study used one experiment class and students were given special treatment or teaching for some time. This treatment is the use of ongoing assessment in collaborative teamwork learning using flip chart on students’. The research data was taken from the value of observations and initial and final tests whose results were tested using the simple linear regression. Based on the results of the first hypothesis and the second hypothesis testing using simple linear regression with Asymp.Sig. (2-tailed) are both obtained 0,00 so H0 is rejected, which means the effect of ongoing assesment in collaborative teamwork learning using flip chart format influences the increase of collaborative abilities and cognitive skills. based on the value of the correlation coefficient (r) in the simple linear regression test obtained 0.42 for the first hypothesis test which means, the effect of ongoing assessment incollaborative teamwork learning using flip charts has a good enough effect on collaborative abilities, which is equal to 42%. Whereas in the second hypothesis test the correlation coefficient value (r) obtained in a simple linear regression test obtained 0.90, which means, the effect of ongoing assessment in collaborative teamwork learning using flip charts has a very good effect on cognitive skills, which is equal to 90%
Perception of librarians towards collaborative research in academic libraries in Nigerian Universities
No full textAbstract
The study is a survey of librarians’ perception towards collaborative research in South Eastern Nigeria. Four research questions guided the study, the study adopted descriptive survey method. The population of the study consists of 242 librarians in federal and state university in South East Nigeria. The instrument for data collection was questionnaire, the data collected was analysed using percentages and bar chart. The result showed that librarians are involved in collaborative research, majority of them are motivated to embrace collaborative research because of high publishing cost, and need for specialization within certain area. It was also discovered that collaborative research has increased librarians skills and knowledge and have encouraged division of labour in writing. Despite these benefits and motivation they still encounter challenges like non contribution by some members in the group and struggle over ranking of authorship as affecting collaborative research. Based on these challenges the study suggested that there should be agreement on the contribution of each author, and ranking of authorship base on the level of contribution by the authors in the group as a way forward in collaborative research
- …
