V.V. Khryashchev – Ph.D. (Eng.), Associate Professor, P.G. Demidov Yaroslavl State University

E-mail: vhr@yandex.ru

A.L. Priopov – Dr.Sc. (Eng.), Associate Professor, P.G. Demidov Yaroslavl State University

E-mail: andcat@yandex.ru

V.A. Pavlov – Post-graduate Student,  P.G. Demidov Yaroslavl State University

E-mail: vladimir@1pavlov.com

L.I. Ivanovsky – Post-graduate Student,  P.G. Demidov Yaroslavl State University

E-mail: leon19unknown@gmail.com

The segmentation of objects in Earth remote sensing images is a perspective area in computer vision and machine learning. Most of the tasks in the analysis of satellite images can be automated by modern algorithms which are presented in this article. One of the most popular approaches to the analysis of satellite images is convolutional neural networks.

The aim of the work is to compare modern neural network approaches for the segmentation of satellite images and create a segmentation algorithm for 3 classes of objects: «water resources», «agriculture» and «forest». The training and testing of the algorithm were carried out on the Landsat-8 and PlanetScope image sets using the DSTL markup and its own, which includes the territory of the Russian Federation and was marked up by 3 independent experts in the Supervise application.

For a comparative analysis of neural network approaches for the segmentation of satellite imagery, U Net, SegNet and TLinkNet were chosen. The accuracy of the segmentation algorithm was estimated using the Sorensen coefficient of similarity. The results of the study show that for the problem of segmentation of satellite images, it is worth using the U-Net algorithm, which received an average value of the Sorensen coefficient of 0,75. The smallest coefficient value was 0,45 for the SegNet algorithm. To create a segmentation algorithm for the allocation of 3 classes of objects «agriculture», «water resources» and «forest» was retrained neural network architecture U-Net. The maximum achieved segmentation accuracy was 96,31% for the class «agriculture», which is due to the clarity of the boundaries of agricultural fields. The least segmentation accuracy is treated in the water resources class of objects due to the presence of ice floes and trees in the water area. To achieve high accuracy of the neural network, its own set of images was created, which included satellite images of different resolutions: 30 and 3 meters per pixel. This approach has improved the accuracy of the selection of the boundaries of the object class «agriculture» by 18,28%. The paper presents a description of publicly available sets of satellite images that can be used to train machine learning algorithms.

The results of the work can be applied in the creation of computer vision systems in the tasks of exact farming, the protection of natural resources and urban planning. The results can be the basis for further research in the field of segmentation of objects in digital images based on neural networks.

1. Sayfeddin D., Bulgakov A.G., Kruglova T.N. Neyrosetevaya sistema otslezhivaniya mestopolozheniya dinamicheskogo agenta na baze kvadrokoptera. Inzhenernyy vestnik Dona. 2014. № 1. URL: ivdon.ru/magazine/archive/n1y2014/2293. [in Russian]
2. Plugotarenko N.K., Varnavskiy A.N. Primenenie neyronnykh setey dlya postroeniya modeli prognozirovaniya sostoyaniya gorodskoy vozdushnoy sredy. Inzhenernyy vestnik Dona. 2012. № 4-2. URL: ivdon.ru/magazine/archive/n4p2y2012/1351. [in Russian]
3. Solov'ev R.A., Tel'pukhov D.V., Kustov A.G. Avtomaticheskaya segmentatsiya sputnikovykh snimkov na baze modifitsirovannoy svertochnoy neyronnoy seti UNET. Inzhenernyy vestnik Dona. 2017. № 4. URL: http://ivdon.ru/ru/magazine/ archive/n4y2017/4433.  [in Russian]
4. Khryashchev V., Ivanovsky L., Pavlov V., Ostrovskaya A., Rubtsov A. Comparison of Different Convolutional Neural Network Architectures for Satellite Image Segmentation. 23rd Conference of Open Innovations Association (FRUCT). 2018. P. 172–179.
5. Khryashchev V., Pavlov V., Priorov A., Ostrovskaya A. Deep learning for region detection in high-resolution aerial images. IEEE EastWest Design & Test Symposium (EWDTS). 2018. P. 1–5.
6. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Arxiv.Org. 2015. URL: arxiv.org/abs/1505.04597.
7. Goodfellow Y., Bengio Y., Courville A. Deep Learning. The MIT Press. 2016. P. 800.
8. Van Etten A. You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery. URL: https://arxiv.org/abs/1805.09512.
9. LandSat Database [Elektronnyy resurs]. URL: https://landsat.visibleearth.nasa.gov.
10. ImageNet Database [Elektronnyy resurs]. URL: http://www.image-net.org.
11. DeepGlobe. CVPR 2018 – Satellite Challenge [Elektronnyy resurs]. URL: http://deepglobe.org.
12. Long J., Shelhamer E., Darrell T. Fully convolutional networks for semantic segmentation. CoRR, abs/1411.4038. 2014. URL: https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf.
13. Ohleyer S.      Building   segmentation        on         satellite   images.   URL:      https://project.inria.fr/aerialimagelabeling/files/2018/01/ fp_ohleyer_compressed.pdf.
14. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer, LNCS. 2015. V. 9351. P. 234–341.
15. Basu S., Ganguly S., Mukhopadhyay S., DiBiano R., Karki M., Nemani R. DeepSat A Learning framework for Satellite Imagery. URL: https://bit.csc.lsu.edu/~saikat/publications/sigproc-sp.pdf.
16. Chhor G., Aramburu C.B. Satellite Image Segmentation for Building Detection using U-net. URL: http://cs229.stanford.edu/ proj2017/final-reports/5243715.pdf.
17. Maggiori E., Tarabalka Y., Charpiat G., Alliez P. Can Semantic Labeling Methods Generalize to Any City? The Inria Aerial Image Labeling Benchmark. IEEE International Geoscience and Remote Sensing Symposium (IGARSS). 2017.
18. DSTL Satellite Imagery Feature Detection. Rezhim dostupa: https://www.kaggle.com/c/dstl-satellite-imagery-feature-detection.
19. Ronneberger O., Fischer P., Brox T. U-Net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, LNCS. 2015. V. 9351: P. 234–241.
20. Badrinarayanan V., Kendall A., Cipolla R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017. V. 39(12). P. 2481–2495.
21. Chaurasia A., Culurciello E. LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation. IEEE Visual Communications and Image Processing (VCIP). 2017. URL: https://arxiv.org/pdf/1707.03718.pdf.