Bayesian Minimum Expected Risk Estimation of Distributions for Statistical Learning

In this thesis, the principle of Bayesian estimation is applied directly to distributions such that the estimated distribution minimizes the expectation of some risk that is a functional of the distribution itself. Bregman divergence is considered as a risk function. An analysis of distribution-based Bayesian quadratic discriminant analysis (QDA) is presented, and a relationship is shown between the proposed approach and an existing regularized quadratic discriminant analysis approach. A functional definition of Bregman divergence is established and it is shown that Bayesian models are optimal in the expected functional Bregman divergence sense. Based on this analysis two practical classifiers are proposed. BDA7 uses a crossvalidated data dependent prior. Local BDA is a modification of Bayesian QDA to achieve flexible model-based classification, by restricting the inference to local neighborhoods of k samples from each class that are closest to the test sample.