TY - GEN

T1 - Asymptotically minimax regret by Bayes mixtures for non-exponential families

AU - Takeuchi, Jun'Ichi

AU - Barron, Andrew R.

PY - 2013

Y1 - 2013

N2 - We study the problems of data compression, gambling and prediction of a sequence xn = x1x2...xn from an alphabet X, in terms of regret with respect to various families of probability distributions. It is known that the regret of the Bayes mixture with respect to a general exponential families asymptotically achieves the minimax value when variants of Jeffreys prior are used, under the condition that the maximum likelihood estimate is in the interior of the parameter space. We discuss a modification of Jeffreys prior which has measure outside the given family of densities, to achieve minimax regret with respect to non-exponential type families, e.g. curved exponential families and mixture families. These results also provide characterization of Rissanen's stochastic complexity for those classes.

AB - We study the problems of data compression, gambling and prediction of a sequence xn = x1x2...xn from an alphabet X, in terms of regret with respect to various families of probability distributions. It is known that the regret of the Bayes mixture with respect to a general exponential families asymptotically achieves the minimax value when variants of Jeffreys prior are used, under the condition that the maximum likelihood estimate is in the interior of the parameter space. We discuss a modification of Jeffreys prior which has measure outside the given family of densities, to achieve minimax regret with respect to non-exponential type families, e.g. curved exponential families and mixture families. These results also provide characterization of Rissanen's stochastic complexity for those classes.

UR - http://www.scopus.com/inward/record.url?scp=84893319176&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84893319176&partnerID=8YFLogxK

U2 - 10.1109/ITW.2013.6691254

DO - 10.1109/ITW.2013.6691254

M3 - Conference contribution

AN - SCOPUS:84893319176

SN - 9781479913237

T3 - 2013 IEEE Information Theory Workshop, ITW 2013

BT - 2013 IEEE Information Theory Workshop, ITW 2013

T2 - 2013 IEEE Information Theory Workshop, ITW 2013

Y2 - 9 September 2013 through 13 September 2013

ER -