R.M. Nemkov - Post-graduate Student, Department of Information Systems and Technologies, North-Caucasus Fed-eral University (Stavropol). E-mail: nemkov.roman@yandex.ru O.S. Mezentseva - Ph.D. (Phys.-Math.), Associate Professor, Professor, Department of Information Systems and Technologies, North-Caucasus Federal University (Stavropol). E-mail: mos@ncstu.ru

In the article a new approach to the creating of noise by the changing of receptive fields (RF) is researched. In the article the theoretical moments of this way and peculiarities of program realization are described. The quality of generalization ability of this method to create of noise is researched. There are comparative characteristics of generalization errors in this article. The problem of invariance is the main and yet unsolved problem of pattern recognition. The different ways of noise are used for the pur-pose to solve this problem. If some internal properties of convolutional neural network (CNN) are changed, for example RFs, then we will obtain the new approach to the creating of noise. In the article describes the theoretical moments connected with the creating of noise by changing the RFs. The place of proposed way in the range of noise creating is pointed out. The high compatibility of this approach with practically any ways of noise is told about. It is paid attention to some tie between the changing of form RF and the real plasticity of RF in biological systems. In this article shortly describes some not trivial moments which can be difficult for programming. This technology is applied to MNIST (standard data set for handwritten digits). Initial conditions of net-s learning, net-s architecture, used algorithm, pools of RFs, the strategies of markup of convolutional layers and etc. are described. The obtained generalization error (1.1%) is compared with error by training without noise and with other errors by using other networks and algorithms. The conclusion for generalization ability is made. Generalization abilities of different scheme of distortion are compared. The conclusion is that the successive distortions on different convolutional layers are more effective than distortion on input layer.

 

1. Dean J., Corrado G.S., Monga R., Chen K., Devin M., Le Q.V., Mao M.Z., Ranzato M.A., Senior A., Tucker P., Yang K., Ng A.Y. Large Scale Distributed Deep Networks // NIPS. 2012.
2. Hinton G.E., Srivastava N., Krizhevsky A., Sutskever I., Salakhutdinov R. Improving Neural Networks by Preventing Co-adaptation of Feature Detectors // CoRR. 2012.
3. Wan L., Zeiler M.D., Zhang S., LeCun Y., Fergus R. Regularization of Neural Networks using DropConnect // ICML 3. V. 28 of JMLR Proceedings. P. 1058−1062.
4. LeCun Y., Bottou L., Bengio Y., Haffner P. Gradient-based Learning Applied to Document Recognition // Proceedings of the IEEE. V. 86. P. 2278−2324. 1998.
5. Decoste D., Scholkopf B. Training Invariant Support Vector Machines // Machine Learning Journal. V. 46. № 1−3. 2002.
6. Simard P., Steinkraus D., Platt J.C. Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis // IEEE Computer Society. 2003. ICDAR. P. 958−962.
7. Ciresan D., Meier U., Schmidhuber J. Multicolumn Deep Neural Networks for Image Classification // Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. CVPR-2012. P. 3642−3649. Washington, DC, USA. IEEE Computer Society.
8. Nemkov R., Mezentseva O. The Use of Convolutional Neural Networks with Nonspecific Receptive Fields // The 4-th International Scientific Conference «Applied Natural Science 2013». Novy Smokovec, High Tatras, Slovak Republic, Oktober 2−4, 2013. P. 148.
9. LeCun Y., Boser B., Denker J.S., Henderson D., Howard R.E., Hubbard W., Jackel L.D. Applied to Hadwritten Zip Code Recognitiion // Neural Computation. 1989. V. 1. № 4. P. 541551.
10. Eysel U.T. Pharmacological studies on receptive field architecture. The cat primary visual cortex / Ed. B. Payne, A. Peters. N.Y.: Acad. Press. 2002. P. 427−470.
11. Larochelle H., Erhan D., Courville A., Bergstra J., Bengio Y. An empirical evaluation of deep architectures on problems with many factors of variation // In Twenty-fourth International Conference on Machine Learning (ICML, 2007).
12. Gonsales R., Vuds R. Cifrovaja obrabotka izobrazhenijj. Izd. 3‑e. M.: Tekhnosfera. 2012. 1104 s.
13. Nemkov R.M., Mezenceva O.S. EHksperimentalnoe issledovanie i analiz vlijanija bazovykh parametrov svertochnykh nejjronnykh setejj na kachestvo ikh obuchenija // Vestnik Severo-Kavkazskogo federalnogo universiteta. 2013. № 3(36) S. 21−26.
14. Nemkov R.M., Mezenceva O.S., Mezencev D.V. Issledovanie vlijanija vosprinimajushhikh svojjstv svertochnykh nejjronnykh setejj na kachestvo raspoznavanija patternov // Infokommunikacionnye tekhnologii v nauke, proizvodstve i obrazovanii. 2014. CHastII. S. 187−193.
15. Modified National Institute of Standards and Technology (MNIST). URL: http://yann.lecun.com/exdb/mnist/.
16. Gusev A.L., CHerepanov F.M., JAsnickijj L.N. Funkcionalnaja predobrabotka vkhodnykh signalov nejjronnojj seti // Nejjrokompjutery: razrabotka, primenenie. 2013. № 5. S. 19−21.
17. Farkhadov M.P., Petukhova N.V., Vaskovskijj S.V., Smirnov V.A. Arkhitektura i kharakteristika sistem raspoznavanija rechi // Nejjrokompjutery: razrabotka, primenenie. 2013. № 12. S. 22−30.