What would judgment and decision making research be like if we took a Bayesian approach to hypothesis testing?

• Main Author: William J. Matthews
• Format: Article
• Language: English
• Published: Society for Judgment and Decision Making 2011-12-01
• Series: Judgment and Decision Making
• Subjects: Null hypothesis significance testing; Bayesian inference; Bayes factor; AnchoringNAKeywords
• Online Access: http://journal.sjdm.org/11/m19/m19.pdf

Judgment and decision making research overwhelmingly uses null hypothesis significance testing as the basis for statistical inference. This article examines an alternative, Bayesian approach which emphasizes the choice between two competing hypotheses and quantifies the balance of evidence provided by the data---one consequence of which is that experimental results may be taken to strongly favour the null hypothesis. We apply a recently-developed ``Bayesian $t$-test'' to existing studies of the anchoring effect in judgment, and examine how the change in approach affects both the tone of hypothesis testing and the substantive conclusions that one draws. We compare the Bayesian approach with Fisherian and Neyman-Pearson testing, examining its relationship to conventional $p$-values, the influence of effect size, and the importance of prior beliefs about the likely state of nature. The results give a sense of how Bayesian hypothesis testing might be applied to judgment and decision making research, and of both the advantages and challenges that a shift to this approach would entail.