63 research outputs found
Systematic reviews to evaluate diagnostic tests.
No full textDiagnostic testing and screening is a critical part of the clinical process because inappropriate diagnostic strategies put patients at risk and entail a serious waste of resources. It is being increasingly recognised that absence of clear summaries of individual research studies on the repeatability, accuracy and impact of tests, which are often scattered across many different journals, is a major impediment. Just as the need to develop means to systematically review research assessing the effectiveness of treatments has been pursued over the last decade, so more recently attention has focused on how research on diagnostic tests might also be systematically reviewed. These reviews present a huge methodological challenge. This paper describes the use of a systematic approach to collation, appraisal and synthesis of information contained in the primary literature about accuracy of diagnostic strategies
Risk prediction models for pressure injury occurrence: An Umbrella Review protocol
No full textAn umbrella review of systematic reviews of risk prediction models for pressure injury occurrence in order to:
a. identify and describe available risk prediction models, their content and development and validation methods used,
b. evaluate the prognostic accuracy of risk prediction models, and
c. evaluate the clinical effectiveness of risk prediction models
Risk prediction models for pressure injury occurrence: An Umbrella Review protocol
No full textAn umbrella review of systematic reviews of risk prediction models for pressure injury occurrence in order to:
a. identify and describe available risk prediction models, their content and development and validation methods used,
b. evaluate the prognostic accuracy of risk prediction models, and
c. evaluate the clinical effectiveness of risk prediction models
A Methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy
No full textOBJECTIVES: To review how heterogeneity has been examined in systematic reviews of diagnostic test accuracy studies. DATA SOURCES: Centre for Reviews and Dissemination's Database of Abstracts of Reviews of Effects (DARE). REVIEW METHODS: Systematic reviews that evaluated a diagnostic or screening test by including studies that compared a test with a reference test were identified from DARE. Reviews for which structured abstracts had been written up to December 2002 were screened for inclusion. Data extraction was undertaken using standardised data extraction forms. RESULTS: A total of 189 systematic reviews met the inclusion criteria. The median number of studies included was 18. Meta-analyses have a higher number with a median of 22 studies compared with 11 for narrative reviews. Graphical plots to demonstrate the spread in study results were provided in 56% of meta-analyses; in 79% these were plots of sensitivity and specificity in the receiver operating characteristic (ROC) space. Statistical tests to identify heterogeneity were used in 32% of reviews: 41% of meta-analyses and 9% of reviews using narrative syntheses. The chi-squared test and Fisher's exact test to assess heterogeneity in individual aspects of test performance were the most common. In contrast, only 16% of meta-analyses used correlation coefficients to test for a threshold effect. A narrative synthesis was used in 30% of reviews. Of the meta-analyses, 52% carried out statistical pooling alone, 18% conducted only summary receiver operator characteristic (SROC) analyses and 30% used both methods of statistical synthesis. For those undertaking SROC analyses, the main differences between the models used were the weights chosen for the regression models, although in 42% of cases the use of, or choice of, weight was not provided. The proportion of reviews using statistical pooling alone has declined from 67% in 1995 to 42% in 2001, with a corresponding increase in the use of SROC methods, from 33% to 58%. However, two-thirds of those using SROC methods also carried out statistical pooling rather than presenting only SROC models. Reviews using SROC analyses also tended to present their results as some combination of sensitivity and specificity rather than using alternative, perhaps less clinically meaningful, means of data presentation such as diagnostic odds ratios. Three-quarters of meta-analyses attempted to investigate statistically possible sources of variation, using subgroup analysis or regression analysis. The impact of clinical or socio-demographic variables was investigated in 74% of these reviews and test- or threshold-related variables in 79%. At least one quality-related variable was investigated in 63% of reviews. Within this subset, the most commonly considered variables were the use of blinding, sample size, the reference test used and the avoidance of verification bias. CONCLUSIONS: The emphasis on pooling individual aspects of diagnostic test performance and the under-use of statistical tests and graphical approaches to identify heterogeneity perhaps reflect the uncertainty in the most appropriate methods to use and also greater familiarity with more traditional indices of test accuracy. This indicates the difficulty and complexity of carrying out such reviews. In these cases it is strongly suggested that meta-analyses are carried out with the involvement of a statistician familiar with the field. Further methodological work on the statistical methods available for combining diagnostic test accuracy studies is needed, as are sufficiently large, prospectively designed primary studies of diagnostic test accuracy comparing two or more tests for the same target disorder. Use of individual patient data meta-analysis in diagnostic test accuracy reviews should be explored to allow heterogeneity to be considered in more detail
Evaluating non-randomised intervention studies
No full textBackground
In the absence of randomised controlled trials (RCTs), healthcare practitioners and policy-makers rely on non-randomised studies to provide evidence of the effectiveness of healthcare interventions. However, there is controversy over the validity of non-randomised evidence, related to the existence and magnitude of selection bias.
Objectives
To consider methods and related evidence for evaluating bias in non-randomised intervention studies.
Methods
1. Three reviews were conducted to consider:
empirical evidence of bias associated with non-randomised studies
the content of quality assessment tools for non-randomised studies
the use of quality assessment in systematic reviews of non-randomised studies.
These reviews were conducted systematically, identifying relevant literature through comprehensive searches across electronic databases, handsearches and contact with experts.
2. New empirical investigations were conducted generating non-randomised studies from two large, multicentre RCTs by selectively resampling trial participants according to allocated treatment, centre and period. These were used to examine:
systematic bias introduced by the use of historical and non-randomised concurrent controls
whether results of non-randomised studies are more variable than results of RCTs
the ability of case-mix adjustment methods to correct for selection bias introduced by non-random allocation.
The resampling design overcame particular problems of meta-confounding and variability of direction and magnitude of bias that hinder the interpretation of previous reviews.
Results
Empirical comparisons of randomised and non-randomised evidence
Eight studies compared results of randomised and non-randomised studies across multiple interventions using meta-epidemiological techniques. The studies reached conflicting conclusions, explicable by differences in:
whether data were sourced from primary studies or systematic reviews
consideration of meta-confounding
inclusion of studies of varying quality
criterion for classifying discrepancies in results.
The only deducible conclusions were (a) results of randomised and non-randomised studies sometimes, but not always, differ and (b) both similarities and differences may often be explicable by other confounding factors.
Quality assessment tools for evaluating non-randomised studies
We identified 194 tools that could be or had been used to assess non-randomised studies. Around half were scales and half checklists, most were published within systematic reviews and most were poorly developed with scant attention paid to principles of scale development.
Sixty tools covered at least five of six pre-specified internal validity domains (creation of groups, blinding, soundness of information, follow-up, analysis of comparability, analysis of outcome), although the degree of coverage varied. Fourteen tools covered three of four core items of particular importance for non-randomised studies (How allocation occurred? Was the study designed to generate comparable groups? Were prognostic factors identified? Was case-mix adjustment used?). Six tools were thought suitable for use in systematic reviews.
Use of quality assessment in systematic reviews of non-randomised studies
Of 511 systematic reviews that included non-randomised studies, only 169 (33%) assessed study quality. Many used quality assessment tools designed for RCTs or developed by the authors themselves, and did not include key quality criteria relevant to non-randomised studies. Sixty-nine reviews investigated the impact of quality on study results in a quantitative manner.
Empirical estimates of bias associated with non-random allocation
The bias introduced by non-random allocation was noted to have two components. First, the bias could lead to consistent over- or underestimations of treatment effects. This occurred for historical controls, the direction of bias depending on time trends in the case-mix of participants recruited to the study. Second, the bias increased variation in results for both historical and concurrent controls, owing to haphazard differences in case-mix between groups. The biases were large enough to lead studies falsely to conclude significant findings of benefit or harm.
Empirical evaluation of case-mix adjustment methods
Four strategies for case-mix adjustment were evaluated: none adequately adjusted for bias in historically and concurrently controlled studies. Logistic regression on average increased bias. Propensity score methods performed better, but were not satisfactory in most situations. Detailed investigation revealed that adequate adjustment can only be achieved in the unrealistic situation when selection depends on a single factor. Omission of important confounding factors can explain underadjustment. Correlated misclassifications and measurement error in confounding variables may explain the observed increase in bias with logistic regression, as may differences between conditional and unconditional odds ratio estimates of treatment effects.
Conclusions
Results of non-randomised studies sometimes, but not always, differ from results of randomised studies of the same intervention. Non-randomised studies may still give seriously misleading results when treated and control groups appear similar in key prognostic factors. Standard methods of case-mix adjustment do not guarantee removal of bias. Residual confounding may be high even when good prognostic data are available, and in some situations adjusted results may appear more biased than unadjusted results.
Although many quality assessment tools exist and have been used for appraising non-randomised studies, most omit key quality domains. Six tools were considered potentially suitable for use in systematic reviews, but each requires revision to cover all relevant quality domains.
Healthcare policies based upon non-randomised studies or systematic reviews of non-randomised studies may need re-evaluation if the uncertainty in the true evidence base was not fully appreciated when policies were made.
The inability of case-mix adjustment methods to compensate for selection bias and our inability to identify non-randomised studies which are free of selection bias indicate that non-randomised studies should only be undertaken when RCTs are infeasible or unethical
Patterns of diffusion: The 1886-1888 measles epidemic and the expansion of settler influence in the Central Interior of British Columbia.
No full textPatterns of Diffusion' argues that the measles epidemic of 1886-1889 was a pivotal event in the indigenous history of British Columbia especially in its Central Interior. This conclusion is drawn primarily through the examination of Hudson's Bay Company (HBC) and Department of Indian Affairs (DIA) documents, but also from newspaper accounts, the oral histories of Imbert Orchard, and the anthropological notes of Marius Barbeau. Prior to this study, no academic work has fully examined the epidemic. Patterns of Diffusion' traces the spread of the epidemic, explores the involvement of the HBC, and examines the origins of the Skeena River Uprising, in which the epidemic was deeply involved. The incorporation of the 1886-1889 measles epidemic into the broader historical narrative contributes to our understanding of the expansion of white settlement and colonial authority in BC's Central Interior region at the end of the nineteenth century. --Leaf ii.The original print copy of this thesis may be available here: http://wizard.unbc.ca/record=b189094
Remits, roles and working models for trial steering committees and data monitoring committees in studies evaluating diagnostic tests: a survey of current practice
No full textSimulation modelling to identify optimal monitoring strategies: the use of the elf biomarker in liver disease monitoring
No full textMethods and mechanisms for measuring and monitoring outcomes from newborn screening: a scoping review
No full textBackground: Newborn screening programmes across the world screen for various rare diseases in newborns, often using a newborn blood spot (NBS) test. Current research is considering use of genomic testing as a screening strategy. In the United Kingdom (UK), newborns are screened for nine rare genetic conditions using an NBS test. Whilst data on process measures (number screened, timeliness of screening, yield, etc.) confirms that the UK NBS programme is operating efficiently, the net benefit on patients and their families is less clear. There is also a lack of evidence to inform decisions regarding candidates for additions to current screening programmes. Outcomes associated with screening programmes that could be measured range from epidemiological outcomes such as incidence and prevalence to natural history outcomes tracking the course of disease, test accuracy, and clinical and educational outcomes following treatment or surveillance. Due to challenges in conducting randomised controlled trials (RCTs) for rare diseases, most studies evaluating relevant outcomes are likely to be observational, so it is important to identify appropriate methods and mechanisms that could be used to collect outcome data. To understand which methods may be most appropriate, we must first understand which methods are currently being used.
Aim: To conduct a scoping review of the literature to identify methods and mechanisms used to measure and monitor outcomes from existing or candidate newborn screening programmes. Our review objectives are to summarise evidence on the following:
• the study designs, their respective objectives and data sources used
• the populations in which the outcomes (short term and long term) have been assessed
• the outcomes included in the relevant studies, including outcomes evaluated in older children, adolescents and adults.
The scoping review will form part of a two-phase project. The scoping review is the first phase which is descriptive in nature to identify the breadth of available evidence and will inform the second phase of the project. In the second phase, relevant methods and mechanisms identified in the scoping review will be evaluated.
Methods: This scoping review will be structured according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR). A search strategy will be developed by an experienced information specialist. The SPIDER framework (Sample, Phenomenon of Interest, Design, Evaluation, Research type) as specified by the UK National Screening Committee (NSC) will be used to determine study eligibility. Both title and abstract and full text screening will be performed by one review author and a random sample of 20% will be independently screened in duplicate by a second review author. A data extraction form will be piloted on 5 studies. Data extraction will be conducted by one author, and a random sample of 20% of data extractions will be done independently in duplicate. All results will be described narratively. Methods and mechanisms will be grouped into categories, and we will synthesise evidence based on these categories. Outcomes will be grouped thematically (epidemiological, natural history, test accuracy, clinical, educational) within each methods/mechanism category
Methods and mechanisms for measuring and monitoring outcomes from newborn screening: a scoping review
No full textBackground: Newborn screening programmes across the world screen for various rare diseases in newborns, often using a newborn blood spot (NBS) test. Current research is considering use of genomic testing as a screening strategy. In the United Kingdom (UK), newborns are screened for nine rare genetic conditions using an NBS test. Whilst data on process measures (number screened, timeliness of screening, yield, etc.) confirms that the UK NBS programme is operating efficiently, the net benefit on patients and their families is less clear. There is also a lack of evidence to inform decisions regarding candidates for additions to current screening programmes. Outcomes associated with screening programmes that could be measured range from epidemiological outcomes such as incidence and prevalence to natural history outcomes tracking the course of disease, test accuracy, and clinical and educational outcomes following treatment or surveillance. Due to challenges in conducting randomised controlled trials (RCTs) for rare diseases, most studies evaluating relevant outcomes are likely to be observational, so it is important to identify appropriate methods and mechanisms that could be used to collect outcome data. To understand which methods may be most appropriate, we must first understand which methods are currently being used.
Aim: To conduct a scoping review of the literature to identify methods and mechanisms used to measure and monitor outcomes from existing or candidate newborn screening programmes. Our review objectives are to summarise evidence on the following:
• the study designs, their respective objectives and data sources used
• the populations in which the outcomes (short term and long term) have been assessed
• the outcomes included in the relevant studies, including outcomes evaluated in older children, adolescents and adults.
The scoping review will form part of a two-phase project. The scoping review is the first phase which is descriptive in nature to identify the breadth of available evidence and will inform the second phase of the project. In the second phase, relevant methods and mechanisms identified in the scoping review will be evaluated.
Methods: This scoping review will be structured according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR). A search strategy will be developed by an experienced information specialist. The SPIDER framework (Sample, Phenomenon of Interest, Design, Evaluation, Research type) as specified by the UK National Screening Committee (NSC) will be used to determine study eligibility. Both title and abstract and full text screening will be performed by one review author and a random sample of 20% will be independently screened in duplicate by a second review author. A data extraction form will be piloted on 5 studies. Data extraction will be conducted by one author, and a random sample of 20% of data extractions will be done independently in duplicate. All results will be described narratively. Methods and mechanisms will be grouped into categories, and we will synthesise evidence based on these categories. Outcomes will be grouped thematically (epidemiological, natural history, test accuracy, clinical, educational) within each methods/mechanism category
- …
