A. P. Trifonov - Dr. Sc. (Eng.), Professor, Honoured Scientist of the Russian Federation, Head of Department of Radiophysics of Voronezh State University. E-mail: trifonov@phys.vsu.ru, trirad@mail.ru
A. V. Kharin - Post-graduate Student of Department of Radiophysics of Voronezh State University. E-mail: phys.vsu@gmail.com

Algorithms for estimating the number of signals based on the method of maximum likelihood or its modifications were investigated. The quantitative characteristic of the quality the functioning of algorithms for estimating the number of signals was proposed, it is a truncated error probability. It is shown that in the case when the amplitudes of the individual signals are not available, estimate number of signals by maximum likelihood method is inconsistent. A few consistent algorithms for estimating the number of signals based on a modification for the method of maximum likelihood were considered. Analysis of algorithms for estimating the number of signals with known or unknown amplitudes was conducted in terms of the truncated error probability. Statistical modeling of the modified maximum likelihood algorithm was performed. Optimal probability parameters with respect to sense of minimizing the truncated error for the algorithms were found that gave an opportunity to significantly improve their performance.