S.N. Grigoriev, D.A. Masterenko

The number of measurements performed worldwide using electronic measuring devices and digital computing permanently increases. In a number of practical cases the observation results are highly quantized, i.e. random scatter caused by both the observed process and random observation error is comparable to quantization step; moreover, changes of measured process parameter are also comparable to quantization step. This kind of situations are common for the most precise measurements. Thus invention of data processing algorithms in order to lower sampling error is an up to date task. Application of traditional methods of mathematical statistics for highly quantized data processing is questionable. The research proved that maximum likelihood estimates gives more precise results compared to usual least square estimates; the most accurate estimates are results of specific estimates - Pitman type estimates. Properties of Pitman type estimates were studied theoretically and practically for an important case of measurands being concerned as linear statistical model parameters. The result of research proved up to 25% error decrease compared to least squares method.