N.A. Andriyanov1, Y.B. Kamalova2, Moiseev G.V.3

1,2 Financial University under the Government of the Russian Federation (Moscow, Russia)

Problem. Today, there are many cloud solutions for machine and even deep learning tasks. However, it is not always clear how much one can trust the models implemented on the basis of such solutions. It is necessary to test the operation of the cloud service on different images in order to understand how well such a system copes with the task of image recognition.

Target. Explore the performance of the Microsoft Custom Vision cloud service and compare the results with the results obtained using standard Python deep learning libraries.

Results. The study is devoted to a comparative analysis of the use of services for automatic machine learning with the development of basic models of convolutional neural networks in the Python environment in the problem of pattern recognition. It has been shown that standard transfer learning methods without fitting parameters are inferior to solutions available in the cloud, such as Microsoft Custom Vision. This compares different neural network models on data such as pollen grain images and luggage x-rays. It is shown that AutoML algorithms allow achieving maximum accuracy on the test set, while the precision and recall metrics reach values ​​2-3% higher than in classical training. A particularly strong difference is observed when training on unbalanced datasets, where the gain in average accuracy and recall can be up to 10-15%.

Practical significance. Cloud systems provide high results on microscopic and X-ray image data. They also do not require knowledge of a programming language from the user, and accordingly allow a larger number of users to develop intelligent solutions for image classification.

Andriyanov N.A., Kamalova Y.B., Moiseev G.V. Comparison of supervised AutoML and supervised ML methods for solving image recognition problems. Neurocomputers. 2022. V. 24. № 5. Р. 19-27. DOI: https://doi.org/10.18127/j19998554-202205-02 (in Russian)

1. Andriyanov N., Dementiev V., Tashlinskiy A., Danilov A. Machine Learning Technologies for Bakery Management Decisions. 24th International Conference on Digital Signal Processing and its Applications. Moscow. 30 March 2022 - 1 April 2022. id: 180062. Р. 1-6. DOI: 10.1109/DSPA53304.2022.9790767.
2. Koroteev M.V. Obzor nekotoryh sovremennyh tendencij v tehnologii mashinnogo obuchenija. E-Management. 2018. № 1. S. 26–35 (in Russian).
3. Tornede A., Gehring L., Tornede T., et al. Algorithm selection on a meta level. Mach. Learn. 2022. № 3. Р. 1-12.
   DOI: 10.1007/s10994-022-06161-4.
4. Das P., Perrone V., Ivkin N., Bansal T., Karnin Z., Shen H., Shcherbatyi I., Elor Y., Wu W., Zolic A., Lienart T., Tang A., Ahmed A., Faddoul J.B., Jenatton R., Winkelmolen F., Gautier P., Dirac L., Perunicic A., Miladinovic M., Zappella G., Archambeau C., Seeger M., Dutt B., Rouesnel L. Amazon SageMaker Autopilot: a white box AutoML solution at scale [Jelektronnyj resurs]. Rezhim dostupa: https://arxiv.org/abs/2012.08483, data obrashhenija 05.09.2022.
5. Demidovskij A., Tugaryov A., Kashchikhin A., Suvorov A., Tarkan Y., Fedorov M., Gorbachev Y. OpenVINO Deep Learning Workbench: Towards Analytical Platform for Neural Networks Inference Optimization. Journal of Physics Conference Series. 2021. V. 1828. № 1. Р. 012012. DOI: 10.1088/1742-6596/1828/1/012012.
6. Andriyanov N.A. Analysis of the Acceleration of Neural Networks Inference on Intel Processors Based on OpenVINO Toolkit. 2020 Systems of Signal Synchronization, Generating and Processing in Telecommunications. SYNCHROINFO 2020. Svetlogorsk. 1-3 July 2020. id: 91660672020. Р. 1-5. DOI: 10.1109/SYNCHROINFO49631.2020.9166067.
7. Azab E., Nafea M., Shihata L.A., Mashaly M. Machine-Learning-Assisted Simulation Approach for Incorporating Predictive Maintenance in Dynamic Flow-Shop Scheduling. Appl. Sci. 2021. № 11. Р. 11725. DOI: 10.3390/app112411725.
8. Salvaris M., Dean D., Tok W.H. Cognitive Services and Custom Vision: Building and Deploying Artificial Intelligence Solutions on the Microsoft AI Platform. Deep Learning with Azure. 2020. Р. 1-189. DOI: 10.1007/978-1-4842-3679-6_5.
9. Kamalova Ju.B., Andrijanov N.A. Raspoznavanie mikroskopicheskih izobrazhenij pyl'cevyh zeren s pomoshh'ju svertochnoj nejronnoj seti VGG-16. Vestnik Juzhno-Ural'skogo gosudarstvennogo universiteta. Ser. Komp'juternye tehnologii, upravlenie, radiojelektronika. 2022. T. 22. № 3. S. 39-46 (in Russian).
10. Gladkih A.A., Andrijanov N.A., Volkov Al.K., Volkov An.K. Razrabotka algoritmov raspoznavanija opasnyh ob’ektov na rentgenovskih snimkah sistem bezopasnosti. Materialy 23-j Vseross. molodezhnoj nauch. konf. «Aktual'nye problemy fizicheskoj i funkcional'noj jelektroniki». 2020. S. 70-71 (in Russian).
11. Krizhevsky A., Sutskever I., Hinton G. ImageNet Classification with Deep Convolutional Neural Networks. NeurIPS. 2012. Р. 1-9. [Jelektronnyj resurs]. – Rezhim dostupa: https://papers.nips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf, data obrashhenija 06.09.2022.
12. Simonyan K., Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition [Jelektronnyj resurs]. – Rezhim dostupa: https://arxiv.org/abs/1409.1556, data obrashhenija 06.09.2022.
13. Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., Rabinovich A. Going Deeper with Convolutions [Jelektronnyj resurs]. – Rezhim dostupa: https://arxiv.org/abs/1409.4842, дата обращения 06.09.2022.
14. Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions [Jelektronnyj resurs]. – Rezhim dostupa: https://arxiv.org/abs/1610.02357, data obrashhenija 06.09.2022.
15. He K., Zhang X., Ren S., Sun J. Deep Residual Learning for Image Recognition [Jelektronnyj resurs]. – Rezhim dostupa: https://arxiv.org/abs/1512.03385, data obrashhenija 07.09.2022.
16. Buslaev A., Iglovikov V., Khvedchenya E., Parinov A., Druzhinin M., Kalinin A. Albumentations: Fast and Flexible Image Augmentations. Information. 2020. V. 11. № 125. DOI: 10.3390/info11020125.
17. Krasheninnikov V.R., Subbotin A.Ju. Dvazhdy stohasticheskie modeli cilindricheskih izobrazhenij. Radiotehnika. 2018. T. 82. № 6. S. 5-8 (in Russian).