603 research outputs found
Discussion of "Multiple Testing for Exploratory Research" by J. J. Goeman and A. Solari
No full textDiscussion of "Multiple Testing for Exploratory Research" by J. J. Goeman and A. Solari [arXiv:1208.2841]
The sequential rejection principle of familywise error control
No full textClosed testing and partitioning are recognized as fundamental principles of familywise error control. In this paper, we argue that sequential rejection can be considered equally fundamental as a general principle of multiple testing. We present a general sequentially rejective multiple testing procedure and show that many well-known familywise error controlling methods can be constructed as special cases of this procedure, among which are the procedures of Holm, Shaffer and Hochberg, parallel and serial gatekeeping procedures, modern procedures for multiple testing in graphs, resampling-based multiple testing procedures and even the closed testing and partitioning procedures themselves. We also give a general proof that sequentially rejective multiple testing procedures strongly control the familywise error if they fulfill simple criteria of monotonicity of the critical values and a limited form of weak familywise error control in each single step. The sequential rejection principle gives a novel theoretical perspective on many well-known multiple testing procedures, emphasizing the sequential aspect. Its main practical usefulness is for the development of multiple testing procedures for null hypotheses, possibly logically related that are structured in a graph. We illustrate this by presenting a uniform improvement of a recently published procedure. © Institute of Mathematical Statistics, 2010
Minimally adaptive BH: A tiny but uniform improvement of the procedure of Benjamini and Hochberg
No full textWe define an adaptive procedure for control of the false discovery rate that is uniformly more powerful than the procedure of Benjamini and Hochberg. The power gain is tiny, however, and only appreciable for small numbers of hypotheses. We illustrate the new method with the case of two hypotheses, for which so far no procedure was known that controls false discovery rate but not also familywise error rate under positive dependence
Multiple hypothesis testing in genomics
No full textThis paper presents an overview of the current state of the art in multiple testing in genomics data from a user's perspective. We describe methods for familywise error control, false discovery rate control and false discovery proportion estimation and confidence, both conceptually and practically, and explain when to use which type of error rate. We elaborate on the assumptions underlying the methods and discuss pitfalls in the interpretation of results. In our discussion, we take into account the exploratory nature of genomics experiments, looking at selection of genes before or after testing, and at the role of validation experiments. © 2014 John Wiley & Sons, Ltd
Rotation-based Multiple Testing in the Multivariate Linear Model.
No full textPermutation methods are very useful in several scientic elds. They have the advantage of making fewer assumptions about the data and of providing more reliable inferential results. They are also particularly useful in case of high-dimensional problems since they easily account for dependence between tests, thereby allowing for more powerful multiplicity control procedures. Indeed, Westfall and Young's min-p procedure often improves on the Holm procedure by providing more rejections. The advantage of being able to make fewer assumptions about the process generating the data unfortunately involves an inherent limitation in the way a process can be modeled (e.g. through multiple linear models). In this work, we propose a permutation (and rotation) method which allows the inference in the multivariate linear model even in the presence of covariates (i.e. nuisance parameters, i.e. confounders). Also, the method allows for the immediate application of the min-p procedure. We make clear how permutations are a particular case of rotations of the data. Permutation tests are exact, while rotation tests retain exactness under multiple-multivariate linear model with normal errors. When errors are not normal, the rotation tests are weakly exchangeable (i.e. approximated and asymptotically exact). A real application to genetic data is presented and discussed
Permutation-based simultaneous confidence bounds for the false discovery proportion
No full textWhen multiple hypotheses are tested, interest is often in ensuring that the proportion of false discoveries is small with high confidence. In this paper, confidence upper bounds for the false discovery proportion are constructed, which are simultaneous over all rejection cut-offs. In particular, this allows the user to select a set of hypotheses post hoc such that the false discovery proportion lies below some constant with high confidence. Our method uses permutations to account for the dependence structure in the data. So far only Meinshausen (2006) has developed an exact, permutation-based and computationally feasible method for obtaining simultaneous false discovery proportion bounds. We propose an exact method which uniformly improves that procedure. Further, we provide a generalization of the method that lets the user select the shape of the simultaneous confidence bounds; this gives the user more freedom in determining the power properties of the method. Interestingly, several existing permutation methods, such as significance analysis of microarrays and the maxT method of Westfall & Young (1993), are obtained as special cases
Robust testing in generalized linear models by sign flipping score contributions
No full textGeneralized linear models are often misspecified because of overdispersion, heteroscedasticity and ignored nuisance variables. Existing quasi-likelihood methods for testing in misspecified models often do not provide satisfactory type I error rate control. We provide a novel semiparametric test, based on sign flipping individual score contributions. The parameter tested is allowed to be multi-dimensional and even high dimensional. Our test is often robust against the mentioned forms of misspecification and provides better type I error control than its competitors. When nuisance parameters are estimated, our basic test becomes conservative. We show how to take nuisance estimation into account to obtain an asymptotically exact test. Our proposed test is asymptotically equivalent to its parametric counterpart
Three-sided hypothesis testing: Simultaneous testing of superiority, equivalence and inferiority
No full textWe propose three-sided testing, a testing framework for simultaneous testing of inferiority, equivalence and superiority in clinical trials, controlling for multiple testing using the partitioning principle. Like the usual two-sided testing approach, this approach is completely symmetric in the two treatments compared. Still, because the hypotheses of inferiority and superiority are tested with one-sided tests, the proposed approach has more power than the two-sided approach to infer non-inferiority or non-superiority. Applied to the classical point null hypothesis of equivalence, the three-sided testing approach shows that it is sometimes possible to make an inference on the sign of the parameter of interest, even when the null hypothesis itself could not be rejected. Relationships with confidence intervals are explored, and the effectiveness of the three-sided testing approach is demonstrated in a number of recent clinical trials. Copyright © 2010 John Wiley & Sons, Ltd
- …
