V.E. Marley1, A.N. Terekhov2, Yu.A. Gatchin3, V.I. Milushkov4, N.N. Limansky5

1,4,5 Admiral S.O. Makarov State University of Marine and River Fleet (Saint Petersburg, Russia)

2 Saint Petersburg State University (Saint Petersburg, Russia)

3 ITMO National Research University (Saint Petersburg, Russia)

1 vmarley@mail.ru, 2 ant@tercom.ru, 3 gatchin1952@mail.ru, 4 info@sohoware.ru, 5 info@sohoware.ru

Auto encoders are powerful tools for solving anomaly detection tasks due to their ability to learn how to compress and restore normal data. The main idea of using autoencoders is to create models capable of efficiently processing normal data. However, this makes it difficult to restore abnormal indicators, which leads to an increase in reconstruction errors. In this article, three types of autoencoders were investigated – convolutional autoencoder (CAE), variational autoencoder (VAE) and adversarial autoencoder (AAE).

To evaluate the performance of various autoencoder models in the task of detecting anomalies on the MNIST dataset, as well as to identify their advantages and limitations.

The results of the study show that all three models of autoencoders have high performance in detecting anomalies, but there is a decrease in performance and computational costs. CAE showed the best results in terms of speed, but in some cases it was inferior to VAE and AAE in terms of accuracy. Despite the more complex architecture and increased training time, VAE and AAE showed only a slight improvement over CAE.

The simplicity and speed of CAE may be preferable for a number of tasks, while VAE and AAE may be useful in cases where additional capabilities such as generating new data or improved noise tolerance are critical. Differences in the results of reconstruction of autoencoder models and their interpretation open up new opportunities for further research, including the development of hybrid approaches that can combine the strengths of these models.

Marley V.E., Terekhov A.N., Gatchin Yu.A., Milushkov V.I., Limansky N.N. Comparative study of the effectiveness of autoencoders in anomaly detection tasks. Neurocomputers. 2024. V. 26. № 5. Р. 96-106. DOI: https://doi.org/10.18127/j19998554-202405-09 (In Russian)

1. Vanyashkin Yu.Yu., Makarov D.A., Popova I.A., Soboleva E.D. Application of autocoders for removing noises from images. StudNet. 2020. V. 3. №. 10. P. 27. (in Russian)
2. Lazarev A.S., Turovsky F.A., Pivovarov S.A. Autocoder. Collection of articles of the International Scientific and Practical Conference "Innovative technologies in modern science": Ufa: Aeterna. 2017. V. 3. P. 82–84. (in Russian)
3. 3 Kalinin M.O. Prevention of cyberthreats in self-organizing m2m environments in case of a lack of training datasets. Digital economy and Industry 5.0: development in a new reality. 2022. P. 107–128. DOI 10.18720/IEP/2022.3/4. (in Russian)
4. Milushkov V.I., Limansky N.N., Marley V.E. Integrating hybrid semi-supervised and contrastive learning for automatic defect classification in manufacturing data: lmproving product quality control accuracy. Prospects of science. 2024. № 6(177). P. 81–86. (in Russian)
5. Limansky N.N., Milushkov V.I., Marley V.E. Anomaly detection in robotic systems: comparative analysis of ConvLSTM with attention mechanism and traditional approaches. Prospects of Science. 2024. № 6(177). P. 77–80. (in Russian)
6. Kuraedov V.I. Application of machine learning methods for automatic test pattern generation process optimization during circuit design. Neurocomputers. 2024. V. 26. № 1. Р. 14-22. DOI 10.18127/j19998554-202401-02. (In Russian)
7. Volkov A.K., Mironova L.V., Potapova S.E. The use of pretrained neural networks for solving the problem of reverse searching of X-ray images of prohibited items and substances. Civil Aviation High Technologies. 2024. V. 27. № 2. P. 8–24. DOI 10.26467/2079-0619-2024-27-2-8-24. (in Russian)
8. Belyakov A.N., Zhukov V.P., Shirokov M.O. Improving the quality of pattern recognition using a modified variational autoencoder. Materials of the International Scientific and Technical Conference. "The state and prospects of development of electrical and thermal technology". Ivanovo: Ivanovo State Power Engineering University named after V.I. Lenin. 2023. V. 2. P. 362–365. (in Russian)
9. LeCun Y., Cortes C., Burges C.J.C. MNIST handwritten digit database. [Electronic resource] – Access mode: https://yann.lecun.com/ exdb/mnist/, date of reference 15.07.2024.
10. Bergmann P. Löwe S., Fauser M., Sattlegger D., Steger C. Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders. [Electronic resource] – Access mode: https://arxiv.org/pdf/1807.02011, date of reference 15.07.2024.
11. Figurnov M.V., Struminsky K.A., Vetrov D.P. Noise-robust method for training of variational autoencoder. Intelligent systems. Theory and applications. 2017. V. 21. № 2. P. 90–109. (in Russian)
12. Osipov K.N., Zamorenov M.V. Minimization of the Kulback-Leibler information discrepancy in the tasks of automated processing of measuring information. Izvestiya Tula State University. Technical sciences. 2019. № 3. P. 195–200. (in Russian)
13. Sukhan A.A. Generative-adversarial neural networks in the tasks of determining trends. Moscow Economic Journal. 2019. № 6. P. 32. DOI 10.24411/2413-046X-2019-16031. (in Russian)
14. Bogdanov L.Y. The evaluation of performance of binary classifiers based on logistic regression using ROC analysis. Bulletin of the Saratov State Technical University. 2010. V. 4. № 2(50). P. 92–97. (in Russian)
15. Kostin D.V., Sheluhin O.I. Comparison of machine learning algorithms for encrypted traffic classification. T-Comm. 2016. V. 10. № 9. Р. 43–52. (in Russian)