A. V. Proletarsky – Dr.Sc. (Eng.), Professor, Dean of Faculty «Informatics and control systems», Bauman Moscow State Technical University

E-mail: pav@bmstu.ru

D. V. Berezkin – Ph.D. (Eng.), Associate Professor, Department of Computer Systems and Networks, Bauman Moscow State Technical University

E-mail: berezkind@bmstu.ru

V. I. Terekhov – Ph.D. (Eng.), Associate Professor, Department of Information Processing Systems and Management, Bauman Moscow State Technical University

E-mail: terekchow@bmstu.ru

P. A. Sekirin – Master’s Degree Student, Department of Information Processing Systems and Management, Bauman Moscow State Technical University

E-mail: sekirinp@gmail.com

I. E. Sergeev – Master’s Degree Student, Department of Computer Systems and Networks, Bauman Moscow State Technical University

E-mail: ilay19940@gmail.com

V. Yu. Sidorov – Master’s Degree Student, Department of Computer Systems and Networks, Bauman Moscow State Technical University

E-mail: sidorow.valentin@gmail.com

The article is devoted to the development of an approach to the creation of a hybrid intellectual system for determining the location of objects from their photographs. The urgency of the problem to be solved is justified. The review and analysis of existing approaches to its solution have been carried out. To solve this problem the algorithm of the hybrid intellectual system using specialized neural networks and ontological reference books has been proposed. The features of the implementation of the basic modules of the proposed hybrid intellectual system have been considered: image preprocessing and ontological modules. The image preprocessing module recalculates the size of the sides of the processed images and analyzes deviations of the input data and the normalization of the original images. Ontological modules search for objects in the images and tag them using a set of different neural networks, each of which analyzes a certain type of information. At the same time, the universal neural network performs general analysis of the image for the presence of objects that belong to different classes, and the text-analytic neural network performs search, extraction and analysis of text data in photographs. To solve the problem of object classifying in a photograph, it has been proposed to use multilayer convolutional neural networks, for teaching which it has been proposed to use photographs located in open sources with known geodata, for example, Google Maps or Foursquare. The analytical module generates the results of determining the geolocation of objects found in the photograph and gives a probabilistic assessment of the result accuracy. If an ambiguity in the choice of the final solution is encountered, it has been proposed to use the hierarchy analysis method, which allows using expert information to calculate the priority of each alternative solution based on a set of criteria. The priority of the criteria with respect to the goal of the solution is calculated on the basis of pairwise comparisons performed by experts at the learning stage of the hybrid intellectual system.

1. Sredstvo dlya proverki metadannykh fotografii Jeffrey's Image Exif Viewer [Elektronnyj resurs] / URL: http://exif-viewer.com/.
2. Zaharieva M., Zeppelzauer M., Breiteneder C. Automated social event detection in large photo collections // Proc. of the 3rd ACM Conference on International Conference on Multimedia Retrieval. ACM. New York. 2013. P. 167–174.
3. Suslov S.I., Gavrikova K.V. Mesta prityazheniya grazhdan v gorodskom prostranstve na primere Krasnogvardejskogo rajona Sankt-Peterburga // Obshchestvo: politika, ekonomika, pravo. 2017. № 5.
4. Weyand T., Kostrikov I., Philbin J. Planet-photo geolocation with convolutional neural networks // European Conference on Computer Vision. Springer, Cham. 2016. P. 37–55.
5. Street View Static API [Elektronnyj resurs] / URL: https://developers.google.com/maps/documentation/streetview/intro?hl=en.
6. Otkrytaya biblioteka komp'yuternogo zreniya OpenCV [Elektronnyj resurs] / URL: https://opencv.org/.
7. Shperber G. Background removal with deep learning [Elektronnyj resurs] // Towards Data Science. 2017. URL: https:// towardsdatascience.com/background-removal-with-deep-learning-c4f2104b3157 (data obrashcheniya: 08.12.2018).
8. Seif G. Semantic segmentation with deep learning [Elektronnyj resurs] // Towards Data Science. 2018. URL: https://towards datascience.com/semantic-segmentation-with-deep-learning-a-guide-and-code-e52fc8958823 (data obrashcheniya: 08.12.2018).
9. Eitel A., Springenberg J.T., Spinello L., Riedmiller M., Burgard W. Multimodal deep learning for robust RGB-D object recognition [Elektronnyj resurs] // Cornell University Library – arXiv.org. URL: https://arxiv.org/pdf/1507.06821.pdf (data obrashcheniya 08.12.2018).
10. Szegedy C. et al. Going deeper with convolutions // Proceedings of the IEEE Conference on Computer Vision and pattern recognition. 2015. P. 1–9.
11. Andreev A.M., Berezkin D.V., Simakov K.V. Metod obucheniya modeli izvlecheniya znanij iz estestvenno-yazykovykh tekstov // Vestnik Moskovskogo gosudarstvennogo tekhnicheskogo universiteta im. N.E. Baumana. Ser. «Priborostroenie». 2007. № 3.
12. Berezkin D.V. Metod avtomatizirovannogo izvlecheniya znanij iz slabostrukturirovannykh istochnikov i ego prime-nenie dlya sozdaniya korporativnykh informatsionnykh sistem [Elektronnyj resurs] / URL: http://www.raai.org/resurs/papers/kolomna2009/ doklad/Berezkin.doc.
13. Andreev A.M., Berezkin D.V., Brik A.V., Smirnov Yu.M. Ispol'zovanie statisticheskikh metodov dlya sozdaniya lingvisticheskogo obespecheniya informatsionno-poiskovoj sistemy // Vestnik Moskovskogo gosudarstvennogo tekhnicheskogo universiteta im. N.E. Baumana. Ser. «Priborostroenie». 2001. № 2.
14. Svidetel'stvo o gos. registratsii programm dlya EVM № 2014617202. Sistema privyazki sobytij, upominaemykh v tekstakh dokumentakh, k geograficheskim ob"ektam / A.M. Andreev, D.V. Berezkin, K.V. Simakov. Zayavl. 23.05.2014.
15. Andreev A.M. i dr. Ispol'zovanie tekhnologii Semantic Web v sisteme poiska nesootvetstvij v tekstakh dokumentov [Elektronnyj resurs] / URL: http://www. inteltec.ru/publish/articles/textan/rimar_RCDL2006.shtml.
16. Berezkin D.V. Tekhnologiya upravleniya raznorodnymi znaniyami // Materialy 2-go Mezhdunar. Pospelovskogo simpoziuma «Gibridnye i sinergeticheskie intellektual'nye sistemy» / Pod red. A.V. Kolesnikova. Kaliningrad: Izd-vo BFU im. I. Kanta. 2014. S. 45–53.
17. Xu W.J., Dong Y.C., Xiao W.L. Is it reasonable for Saaty's consistency test in the pairwise comparison method? // ISECS International Colloquium on Computing, Communication, Control, and Management. 2008. IEEE. 2008. V. 3. P. 294–298.
18. Riza S.P., Murtuzayeva M. Application Saaty pair comparisons method to the investments distribution in parameters of ecological sustainability // Problems of Cybernetics and Informatics (PCI). 2012 IV International Conference. IEEE. 2012. P. 1–3.
19. Tang K. et al. Improving image classification with location context // Proceedings of the IEEE International Conference on Computer Vision. 2015. P. 1008–1016.
20. Hays J., Efros A.A. IM2GPS: estimating geographic information from a single image // IEEE Conference on Computer Vision and Pattern Recognition. 2008. P. 1–8.