An Inferential Measure of Dependence Between Two Systems Using Bayesian Model Comparison

Overview & Synopsis

We propose to quantify dependence between two systems X and Y in a dataset D based on the Bayesian comparison of two models: one, H0, of statistical independence and another one, H1, of dependence. In this framework, dependence between X and Y in D, denoted B(X , Y|D), is quantified as P (H1|D), the posterior probability for the model of dependence given D, or any strictly increasing function thereof. It is therefore a measure of the evidence for dependence between X and Y as modeled by H1 and observed in D. We review several statistical models and reconsider standard results in the light of B(X , Y|D) as a measure of dependence. Using simulations, we focus on two specific issues: the effect of noise and the behavior of B(X , Y|D) when H1 has a parameter coding for the intensity of dependence. We then derive some general properties of B(X , Y|D), showing that it quantifies the information contained in D in favor of H1 versus H0. While some of these properties are typical of what is expected from a valid measure of dependence, others are novel and naturally appear as desired features for specific measures of dependence, which we call inferential. We finally put these results in perspective; in particular, we discuss the consequences of using the Bayesian framework as well as the similarities and differences between B(X , Y|D) and mutual information.