1,721,188 research outputs found
Case-control and two-gate designs in diagnostic accuracy studies
No full textBACKGROUND: In some diagnostic accuracy studies, the test results of a series of
patients with an established diagnosis are compared with those of a control
group. Such case-control designs are intuitively appealing, but they have also
been criticized for leading to inflated estimates of accuracy.
METHODS: We discuss similarities and differences between diagnostic and etiologic
case-control studies, as well as the mechanisms that can lead to variation in
estimates of diagnostic accuracy in studies with separate sampling schemes
("gates") for diseased (cases) and nondiseased individuals (controls).
RESULTS: Diagnostic accuracy studies are cross-sectional and descriptive in
nature. Etiologic case-control studies aim to quantify the effect of potential
causal exposures on disease occurrence, which inherently involves a time window
between exposure and disease occurrence. Researchers and readers should be aware
of spectrum effects in diagnostic case-control studies as a result of the
restricted sampling of cases and/or controls, which can lead to changes in
estimates of diagnostic accuracy. These spectrum effects may be advantageous in
the early investigation of a new diagnostic test, but for an overall evaluation
of the clinical performance of a test, case-control studies should closely mimic
cross-sectional diagnostic studies.
CONCLUSIONS: As the accuracy of a test is likely to vary across subgroups of
patients, researchers and clinicians might carefully consider the potential for
spectrum effects in all designs and analyses, particularly in diagnostic accuracy
studies with differential sampling schemes for diseased (cases) and nondiseased
individuals (controls)
Variation of a test's sensitivity and specificity with disease prevalence
No full textBACKGROUND:Anecdotal evidence suggests that the sensitivity and specificity of a diagnostic test may vary with disease prevalence. Our objective was to investigate the associations between disease prevalence and test sensitivity and specificity using studies of diagnostic accuracy. METHODS:We used data from 23 meta-analyses, each of which included 10-39 studies (416 total). The median prevalence per review ranged from 1% to 77%. We evaluated the effects of prevalence on sensitivity and specificity using a bivariate random-effects model for each metaanalysis, with prevalence as a covariate. We estimated the overall effect of prevalence by pooling the effects using the inverse variance method. RESULTS:Within a given review, a change in prevalence from the lowest to highest value resulted in a corresponding change in sensitivity or specificity from 0 to 40 percentage points. This effect was statistically significant (p < 0.05) for either sensitivity or specificity in 8 metaanalyses (35%). Overall, specificity tended to be lower with higher disease prevalence; there was no such systematic effect for sensitivity. INTERPRETATION:The sensitivity and specificity of a test often vary with disease prevalence; this effect is likely to be the result of mechanisms, such as patient spectrum, that affect prevalence, sensitivity and specificity. Because it may be difficult to identify such mechanisms, clinicians should use prevalence as a guide when selecting studies that most closely match their situation
A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard
No full textOBJECTIVE: In diagnostic accuracy studies, the reference standard may be
imperfect or not available in all patients. We systematically reviewed the
proposed solutions for these situations and generated methodological guidance.
STUDY DESIGN AND SETTING: Review of methodological articles.
RESULTS: We categorized the solutions into four main groups. The first group
includes methods that impute or adjust for missing data on the reference
standard. The second group consists of methods that correct estimates of accuracy
obtained with an imperfect reference standard. In the third group a reference
standard is constructed by combining multiple test results through a predefined
rule, based on a consensus procedure, or through statistical modeling. In the
fourth group, the diagnostic accuracy paradigm is abandoned in favor of
validation studies that relate index test results to relevant clinical data, such
as history, future clinical events, and response to therapy.
CONCLUSION: Most of the methods try to impute, adjust, or construct a reference
standard. In situations that deviate only marginally from the classical
diagnostic accuracy paradigm, these are valuable methods. In cases where an
acceptable reference standard does not exist, the concept of clinical test
validation may provide an alternative paradigm to evaluate a diagnostic test
Bivariate meta-analysis of predictive values of diagnostic tests can be an alternative to bivariate meta-analysis of sensitivity and specificity
Full text linkOBJECTIVE: Meta-analysis of predictive values is usually discouraged because
these values are directly affected by disease prevalence, but sensitivity and
specificity sometimes show substantial heterogeneity as well. We propose a
bivariate random-effects logitnormal model for the meta-analysis of the positive
predictive value (PPV) and negative predictive value (NPV) of diagnostic tests.
STUDY DESIGN AND SETTING: Twenty-three meta-analyses of diagnostic accuracy were
reanalyzed. With separate models, we calculated summary estimates of the PPV and
NPV and summary estimates of sensitivity and specificity. We compared these
summary estimates, the goodness of fit of the two models, and the amount of
heterogeneity of both approaches.
RESULTS: There were no substantial differences in the goodness of fit or amount
of heterogeneity between both models. The median absolute difference between the
projected PPV and NPV from the summary estimates of sensitivity and specificity
and the summary estimates of PPV and NPV was 1% point (interquartile range, 0-2%
points).
CONCLUSION: A model for the meta-analysis of predictive values fitted the data
from a range of systematic reviews equally well as meta-analysis of sensitivity
and specificity. The choice for either model could be guided by considerations of
the design used in the primary studies and sources of heterogeneity
Bivariate and SROC regression models in meta-analysis of studies of diagnostic accuracy (Oral)
No full textPartial and differential verification in diagnostic accuracy studies Abstract [Oral: O06-1]
No full textUse of methodological search filters to identify diagnostic accuracy studies can lead to the omission of relevant studies
No full textOBJECTIVE: To determine the usefulness of methodological filters in search
strategies for diagnostic studies in systematic reviews.
STUDY DESIGN AND SETTING: We made an inventory of existing methodological search
filters for diagnostic accuracy studies and applied them in PubMed to a reference
set derived from 27 published systematic reviews in a broad range of clinical
fields. Outcome measures were the fraction of not identified relevant studies and
the reduction in the number of studies to read.
RESULTS: We tested 12 search filters. Of the studies included in the systematic
reviews, 2%-28% did not pass the sensitive search filters, 4%-24% did not pass
the accurate filters, and 39%-42% did not pass the specific filters. Decrease in
number-needed-to-read when a search filter was used in a search strategy for a
diagnostic systematic review varied from 0% to 77%.
CONCLUSION: The use of methodological filters to identify diagnostic accuracy
studies can lead to omission of a considerable number of relevant studies that
would otherwise be included. When preparing a systematic review, it may be
preferable to avoid using methodological filters
Systematic reviews of diagnostic tests in cancer: review of methods and reporting
No full textObjectives To assess the methods and reporting of systematic reviews of diagnostic tests. Data sources Systematic searches of Medline, Embase, and five other databases identified reviews of tests used in patients with cancer. Of these, 89 satisfied our inclusion criteria of reporting accuracy of the test compared with a reference test, including an electronic search, and published since 1990. Review methods All reviews were assessed for methods and reporting of objectives, search strategy, participants, clinical setting, index and reference tests, study design, study results, graphs, meta-analysis, quality, bias, and procedures in the review. We assessed 25 randomly selected reviews in more detail. Results 75% (67) of the reviews stated inclusion criteria, 49% (44) tabulated characteristics of included studies, 40% (36) reported details of study design, 17% (15) reported on the clinical setting, 17% (15) reported on the severity of disease in participants, and 49% (44) reported on whether the tumours were primary, metastatic, or recurrent. Of the 25 reviews assessed in detail, 68% (17) stated the reference standard used in the review, 36% (9) reported the definition of a positive result for the index test, and 56% (14) reported sensitivity, specificity, and sample sizes for individual studies. Of the 89 reviews, 61% (54) attempted to formally synthesise results of the studies and 32% (29) reported formal assessments of study quality. Conclusions Reliability and relevance of current systematic reviews of diagnostic tests is compromised by poor reporting and review methods. <br/
Sources of variation and bias in studies of diagnostic accuracy: a systematic review
No full textBACKGROUND: Studies of diagnostic accuracy are subject to different sources of
bias and variation than studies that evaluate the effectiveness of an
intervention. Little is known about the effects of these sources of bias and
variation.
PURPOSE: To summarize the evidence on factors that can lead to bias or variation
in the results of diagnostic accuracy studies.
DATA SOURCES: MEDLINE, EMBASE, and BIOSIS, and the methodologic databases of the
Centre for Reviews and Dissemination and the Cochrane Collaboration. Methodologic
experts in diagnostic tests were contacted.
STUDY SELECTION: Studies that investigated the effects of bias and variation on
measures of test performance were eligible for inclusion, which was assessed by
one reviewer and checked by a second reviewer. Discrepancies were resolved
through discussion.
DATA EXTRACTION: Data extraction was conducted by one reviewer and checked by a
second reviewer.
DATA SYNTHESIS: The best-documented effects of bias and variation were found for
demographic features, disease prevalence and severity, partial verification bias,
clinical review bias, and observer and instrument variation. For other sources,
such as distorted selection of participants, absent or inappropriate reference
standard, differential verification bias, and review bias, the amount of evidence
was limited. Evidence was lacking for other features, including incorporation
bias, treatment paradox, arbitrary choice of threshold value, and dropouts.
CONCLUSIONS: Many issues in the design and conduct of diagnostic accuracy studies
can lead to bias or variation; however, the empirical evidence about the size and
effect of these issues is limited
- …
