Publishing house Radiotekhnika

"Publishing house Radiotekhnika":
scientific and technical literature.
Books and journals of publishing houses: IPRZHR, RS-PRESS, SCIENCE-PRESS

Тел.: +7 (495) 625-9241


On the use of mutual information as objective function for image parameters’ estimation


V.E. Dementiev – Ph.D. (Eng.), associate professor at Ulyanovsk State Technical University. E-mail:
S.V. Voronov – Post-graduate student of Ulyanovsk State Technical University. E-mail:

Recently, when specifying the objective function for digital image parameters’ estimation information-theoretic approach is becoming common. When using stochastic gradient estimation procedure the most interesting information-theoretic objective function is mutual information (MI). A key step in evaluating the MI gradient estimation is to estimate the probability density function (PDF) using image sample. Based on the approach to solve this problem the MI gradient estimation approaches can be divided into two groups. The first group called histogram-based methods. Here entropy and the joint entropy of the images are estimated using the histograms of image samples. In the second group methods are based on the PDF reconstruction using image samples. Methods of the second group, in particular Parzen window density estimation, provide better PDF estimation in comparison with the histogram-based methods. However, their disadvantages are low accuracy of MI gradient estimation and requirement of big image sample. The last one is undesirable in real-time image processing. A method of estimating the entropy of images based on the cross-validation procedure and aimed at reducing the computational cost is proposed. Studies have shown that the application of the proposed method allows to reduce the sample size twice in comparison with the state-of-the-art algorithm EMMA. In this case the computational cost of image entropy estimation reduces by 10-12 % with a slight increase in the error variance (a few percent). When using the Parzen window method the pixel intensity gradients with respect to the spatial coordinates are estimated via the finite differences. Due to image correlation the quality of the estimates and the MI gradient estimates depends on the changes in parameters used for finite difference estimation. The possibilities for error minimization finding image intensities PDF by optimizing the values of these increments are analyzed. It is shown that the optimal values determined by the form of the image correlation function and signal-to-noise ratio. Results of statistical modeling have shown adequacy of the analytical expressions. Using the optimization provides almost double the gain in performance of the stochastic gradient image inter-frame geometric deformation estimation.

  1. Vasil’ev K.K., Dement’ev V.E., Luchkov N.V. Analysis of Efficiency of Detecting Extended Signals on Multidimensional Grids // Pattern Recognition and Image Analysys. 2013. V. 23. № 1. P. 1-10.
  2. Tashlinskij A.G. Oczenivanie parametrov prostranstvenny'x deformaczij posledovatel'nostej. Ul'yanovsk: izdatel'stvo UlGTU. 2000. 132 s.
  3. Taslinskii A.G. Pseudogradient Estimation of Digital Images Interframe Geometrical Deformations // Vision Systems: Segmentation & Pattern Recognition. Vienna, Austria: I-Tech Education and Publishing. 2007. R. 465-494.
  4. Tashlinskij A.G., Voronov S.V. Analiz czelevy'x funkczij pri rekurrentnom oczenivanii mezhkadrovy'x geometricheskix deformaczij izobrazhenij // Naukoemkie texnologii. 2013. T. 14. № 5. S. 16-21.
  5. Tashlinskii A.G., Safina G.L., Voronov S.V. Pseudogradient optimization of objective function in estimation of geometric interframe image deformations // Pattern recognition and image analysis. 2012. V. 22. № 2. P. 386-392.
  6. Tashlinskii A.G., Safina G.L., Voronov S.V. Optimization of mismatch euclidean distance in evaluating interframe deformations of geometrical images // Pattern recognition and image analysis. 2011. V. 21. № 2. P. 335-338.
  7. Tashlinskij A.G., Smirnov P.V., Biktimirov L.Sh. Kriterii ostanovki proczessa psevdogradientnoj privyazki izobrazhenij na osnova analiza sxodimosti oczenok parametrov privyazki // Naukoemkie texnologii. 2013. T. 14. № 5. S. 22-25.
  8. Vasil'ev K.K., Dement'ev V.E. Avtoregressionny'e modeli mnogomerny'x izobrazhenij // Naukoemkie texnologii. 2013. T. 14. № 5. S.12-15.

June 24, 2020
May 29, 2020

© Издательство «РАДИОТЕХНИКА», 2004-2017            Тел.: (495) 625-9241                   Designed by [SWAP]Studio