Yu.A. Bryuchanov, Yu.A. Lukashevich

Errors caused by quantization and overflow effects in digital filtration can be associated with quantizer characteristics, which are essentially nonlinear, thus determining the nature of the processes in digital filters. In practice, two types of overflow quantizer characteristics are used: characteristics with saturation and sawtooth characteristics. The purpose of the investigation is the calculation of the steady periodic oscillations in low-pass Butterworth and Chebyshev (type 1) digital filters when the input is harmonic signal and quantization and saturation effects are taking into consideration. The calculation of filter output spectral components and estimation of nonlinear distortions are also performed. The dependence characteristics of nonlinear distortion from coding method, numbers approximation technique and the number of bits in the numbers representation are given. Calculation of forced periodic oscillations is performed by using a method based on the representation of stationary oscillations in the form of invariant set of discrete nonlinear point maps. The spectral composition of the filter reaction is obtained by applying Discrete Fourier Transform (DFT). The classic formula is used for the integer T, while its modification - for non non-integer T. In order to identify consistent patterns the calculations of nonlinear distortions are performed for digital low-pass Butterworth and Chebyshev type 1 filters, which have a bandwidth ripple of 0.2 dB, with periodic inputs of the form x(n)=Acosωn, where ω=2π/T. The approach is based on the restrictions that the filters use fractional arithmetic, only nonlinear characteristics of adder quantizers are taken into consideration, the overflow characteristics of quantizers have saturation, and the characteristics of the multipliers are linear. The results show that the lowest distortions appear when the rounding of number is used. If number of bits in the number representation R[5,8], the highest distortions in the both types of filters appear when an additional code with a truncation is used, that is probably caused by the quantizer characteristics asymmetry. The coefficient of nonlinear distortions К in the Butterworth filters is less than 10% when the number of bits R ≥ 7 (if the additional code with a truncation is used, R ≥ 8 for the filter order 2G = 8). It should be noted that the level of distortions in Chebyshev digital filters is higher, that is caused by greater bandwidth ripple of magnitude frequency response. If R ≤ 5, the level of distortions in Chebyshev filters is several times higher than the level of distortions in Butterworth filters. The nonlinear distortions coefficient К in the Chebyshev filters is less than 10% when the number of bits R ≥ 7 (if the additional code with a truncation is used, R ≥ 8 for the filter order 2G = 8). Level of distortions increases with the number of filter cascades, that is caused by increasing number of nonlinear signal transformations. Obtained dependence characteristics of nonlinear distortions coefficient K from filter type, numbers representation and quantizer characteristics allow to choose appropriate type and order of the filter and quantizer parameters during the designing of message transmission system with digital signal processing.