Abstract:

We consider a problem of parameters estimation for gaussian mixture models widely used in data analysis and unsupervised machine learning. A new model identification method based on Bayesian aproach and the principle of maximum posterior distribution is proposed. In the article we describe the  method of multiextremum density function maximum definition using sampling by Metropolis-Hastings algorithm. The proposed method is compared with the traditional expectation maximization algorithm by computational experiments both on a sample synthetic data and the real one from <<fisheriris>> dataset.

Keywords:

Gaussian mixture model, Bayesian approach, Metropolis-Hastings algorithm, classification problem.

PP. 101-111.

REFERENCES

1. McLachlan G.J. Mixture Models: inference and applications to clustering. – Marcel Dekker, New York, 1988.
2. McLachlan, G., and D. Peel. Finite Mixture Models. – Hoboken, NJ: John Wiley & Sons. Inc., 2000.
3. Figueiredo, M.A.T. and Jain A.K. Unsupervised Learning of Finite Mixture Models. // IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.24(3), pp.381-396, 2012.
4. Reynolds, D.A., Rose, R.C. Robust Text-Independent Speaker Identification using Gaussian Mixture Speaker Models // IEEE Transactions on Acoustics, Speech, and Signal Processing, vol.3(1). pp.72-83, 1995.
5. Brigo Damiano, Mercurio Fabio. Lognormal-mixture dynamics and calibration to market volatility smiles. // International Journal of Theoretical and Applied Finance, vol.5(4), pp.427-452, 2002.
6. Dempster A.P., Laird N.M., Rubin D.B. Maximum Likelihood from Incomplete Data via the EM Algorithm.// Journal of the References Royal Statistical Society. Series B, vol.39(1), pp.1-38, 1977.
7. Xu L., Jordan M.I. On Convergence Properties of the EM Algorithm for Gaussian Mixtures. // Neural Computation, vol.8(1). pp.129-151, 1996.
8. Christopher M. Bishop. Pattern Recognition and Machine Learning. – Springer, 2006 – 758 p.
9. Korolev V.Yu. EM-algoritm, ego modificacii i primeneniyek zadache razdeleniya smesey veroyatnostnih raspredeleniy. – M.: IPI RAN, 2004. – 102 s.
10. Korolev V.Yu., Nazarov A.L. Razdeleniye smesey veroyatnostnih raspredeleniy pri pomoshi setochnih metodov momentov i maksimalnogo pravdopodobiya // Avtomatica i Telemechanika, vip.3, c.98-116, 2010.
11. Krivenko M.P. Rasshepleniye smesi veroyatnostnih raspredeleniy na dve sostavlyaushiye // Informatica i ee primeneniya, t.2, vip.4, s.48-56, 2008.John E. Rolph. Bayesian Estimation of Mixing Distributions // The Annals of Mathematical Statistics, vol.39, No.4, pp.1289-1302, 1968.
12. John E. Rolph. Bayesian Estimation of Mixing Distributions // The Annals of Mathematical Statistics, vol.39, No.4, pp.1289- 1302, 1968.
13. Alexander Boulytchev, Vladimir Britkov System modeling of regional economic processes dynamic on the base of the information modeling technology // Proceedings Of The 10th Eurasia Business And Economics Society Conference (EBES) (May 23-25, 2013, Istanbul, Turkey). – Istanbul: EBES Publications, 2013. – pp.346-354.
14. Andrew Gelman. Bayes, Jeffreys, Prior Distributions and the Philosophy of Statistics // Statistical Science, vol.24, No.2, pp.176-178, 2009.
15. Robert E. Kass and Larry Wasserman. The Selection of Prior Distributions by Formal Rules // Journal of the American Statistical Association, vol.91, No.435, pp.1343-1370, 1996.
16. Navid Feroze and Muhammad Aslam. Bayesian Estimation of Two-Component Mixture of Gumbel Type II Distribution under Informative Priors // International Journal of Advanced Science and Technology, vol.53, pp.11-30, 2013.
17. Zellner A. Past and Recent Results on Maximal DataInformation Priors // Texhnical Report, Graduate School of Business, University of Chicago, 1996.
18. Britkov V.B., Boulytchev A.V. Informacionnoye modelerovaniye slojnih plohoformalizuemih sistem // Prikladniye problemi upravleniya macrosistemami. Pod redakciyey Yu.S. Popkova, V.A. Putilova. – M.: KRASAND, 2010. – s.216-231. (Trudi Instituta sistemnogo analiza RAN, t.59).
19. Siddhartha Chib, Edward Greenberg. Understanding the Metropolis-Hastings Algorithm // The American Statistician, vol.49, No.4, pp.327-335, 1995.
20. Popkov Y.S., Dubnov Y.A., Popkov A.Y. New Method of Randomized Forecasting Using Entropy-Robust Estimation: Application to the World Population Prediction. // Mathematics, 2016, vol.4, No.16, pp.1-16.
21. Lichman M. UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine. CA: University of California, School of Information and Computer Science, 2013.
22. Kamil Dedecius and Jan Reichl. Distributed Estimation of Mixture Models. – Springer International Publishing, 2015. – pp. 27-36.