posted on 2017-01-04, 15:21authored byAnthony Quinn, MIroslav Kárný, Tatiana V Guy
The minimum cross-entropy principle is an established technique for design of an unknown distribution, processing linear functional constraints on the distribution. More generally, fully probabilistic design (FPD) chooses the distribution—within the knowledge-constrained set of possible distributions—for which the Kullback-Leibler divergence to the designer’s ideal distribution is minimized. These principles treat the unknown distribution deterministically. In this paper, fully probabilistic design is applied to hierarchical Bayesian models for the first time, yielding optimal design of a (possibly nonparametric) stochastic model for the unknown distribution. This equips minimum cross-entropy and FPD distributional estimates with measures of uncertainty. It enables robust choice of the optimal model, as well as randomization of this choice. The ability to process non-linear functional constraints in the constructed distribution significantly extends the applicability of these principles. Currently available FPD procedures for (a) merging of external knowledge, (b) approximate learning and stabilized forgetting, (c) decision strategy design, and (d) local adaptive control design, are unified for the first time via the hierarchical FPD framework of this paper.
History
Publication
Information Sciences;369, pp. 532-547
Publisher
Elsevier
Note
peer-reviewed
Other Funding information
SFI
Rights
This is the author’s version of a work that was accepted for publication in Information Sciences. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Information Sciences, 2016, 369, pp. 532-547, http://dx.doi.org/10.1016/j.ins.2016.07.035