Distributions associated with simultaneous multiple hypothesis testing

• Main Authors: Chang Yu, Daniel Zelterman
• Format: Article
• Language: English
• Published: SpringerOpen 2020-10-01
• Series: Journal of Statistical Distributions and Applications
• Subjects: Bonferroni correction; Simes criteria; False discovery rate; p-values
• Online Access: http://link.springer.com/article/10.1186/s40488-020-00109-6

Abstract We develop the distribution for the number of hypotheses found to be statistically significant using the rule from Simes (Biometrika 73: 751–754, 1986) for controlling the family-wise error rate (FWER). We find the distribution of the number of statistically significant p-values under the null hypothesis and show this follows a normal distribution under the alternative. We propose a parametric distribution Ψ I (·) to model the marginal distribution of p-values sampled from a mixture of null uniform and non-uniform distributions under different alternative hypotheses. The Ψ I distribution is useful when there are many different alternative hypotheses and these are not individually well understood. We fit Ψ I to data from three cancer studies and use it to illustrate the distribution of the number of notable hypotheses observed in these examples. We model dependence in sampled p-values using a latent variable. These methods can be combined to illustrate a power analysis in planning a larger study on the basis of a smaller pilot experiment.