A.I. Vlasov1, N.V. Zavyalov2, K.V. Selivanov3, I.I. Skalchenkov4

1–4 Bauman Moscow State Technical University (Moscow, Russia)

A human operator with a long monotonous activity tends to make mistakes in the work of modern mass production of electronic products, this problem is particularly deep. An important stage of such production is product quality control, a characteristic error in the work of the output control department is the omission of a defective printed circuit board. The most critical mistake is in the production of high-tech products, which makes it impossible to continue using the product, and also completely rejects all radio-electronic elements already sealed on a defective printed circuit board. All this leads to the need to develop modern digital methods and technologies for automatic control of components of electronic products using artificial intelligence. The solution of the problem of studying the possibility of detecting defects on printed circuit boards using neural networks is aimed at creating industrial modern solutions for highly automated quality control systems for printed circuit boards.

The purpose of this work is to develop neural network methods and tools for automatic detection of PCB defects during visual inspection.

The analysis of neural networks designed to work with images is carried out. Based on the results of the analysis, the choice of neural networks on the U-Net and Signet architectures for solving the problem of automatic flaw detection of printed circuit boards is justified. A comparison of the effectiveness of these neural networks on detecting defects of printed circuit boards is carried out, based on the data obtained during the experiment, graphs of the learning process are constructed. The overall results of the two architectures do not have strong differences, but when working on images of a conductive pattern with small elements, the convolutional neural network with the U-Net architecture showed increased efficiency. Software has been developed that allows detecting and classifying basic types of PCB defects. The software implementation of the complex was tested on printed circuit boards of 5 and higher accuracy classes and an example of the operation of the complex on six images is presented.

The proposed implementations of neural networks can be used in the creation of automated information systems (AIS) that allow monitoring and diagnostics of the product at all technological stages of its production. The use of AIS in production makes it possible to reduce the role of the human factor in production, reduce the time spent on quality control.

Vlasov A.I., Zavyalov N.V., Selivanov K.V., Skalchenkov I.I. Application of neural networks in detecting PCB defects. Neurocomputers. 2022. V. 24. № 6. Р. 5-19. DOI: https://doi.org/10.18127/j19998554-202206-01 (In Russian).

1. Makushina N.V., Sergeyeva M.D. Analiz defektov metallizirovannykh otverstiy pechatnykh plat. Proyektirovaniye i tekhnologiya elektronnykh sredstv. 2018. № 1. S. 3–12. (in Russian).
2. Markelov V.V., Kabayeva A.S. Upravleniye kachestvom elektronnykh sredstv. M.: Izd-vo MGTU im. N.E. Baumana. 2014. 272 s. (in Russian).
3. Buyanov A.A., Vlasov A.I., Gridnev V.N. The neuronet hardware and software integrated system for defectoscopy of printed circuits based on microsections. Sb. Trudov 3-y mezhdunar. konf. "Kompyuternyye metody i obratnyye zadachi v nerazrushayushchem kontrole i diagnostike". 2002. S. 71. (in Russian).
4. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv preprint arXiv:1505.04597. 2015.
5. Badrinarayanan V., Kendall A., Cipolla R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. arXiv preprint arXiv:1511.00561v3. 2016.
6. Viryasova A.Yu., Vlasov A.I., Gladkikh A.A. Neyrosetevyye metody defektoskopii integralnykh struktur. Neyrokompyutery: razrabotka. primeneniye 2019. № 2. S. 54–67. (in Russian).
7. Buyanov A.I., Vlasov A.I., Zagoskin A.V. Primeneniye neyrosetevykh metodov pri defektoskopii pechatnykh plat. Neyrokompyutery: razrabotka. primeneniye. 2002. № 3. S. 42–70. (in Russian).
8. Panfilova S.P., Vlasov A.I., Gridnev V.N., Chervinskiy A.S. Beskontaktnyy teplovoy kontrol izdeliy elektronnoy tekhniki. Proizvodstvo elektroniki. 2007. № 3. S. 25–30. (in Russian).
9. Panfilova S.P., Vlasov A.I., Gridnev V.N., Chervinskiy A.S. Beskontaktnyy teplovoy kontrol elektronno-vychislitelnykh sredstv. Tekhnologiya i konstruirovaniye v elektronnoy apparature. 2007. № 6. S. 42–49. (in Russian).
10. Gridnev V.N., Sergeyeva M.D., Chebova A.I. Lineynyye modeli raspoznavaniya teplovizionnykh izobrazheniy neispravnostey elektronnykh yacheyek. Kontrol. Diagnostika. 2014. № 8. S. 57–66. (in Russian).
11. Averianikhin A.E., Vlasov A.I., Evdokimova E.V. Ierarkhicheskaya piramidalnaya subdiskretizatsiya v glubokikh svertochnykh setyakh dlya raspoznavaniya vizualnykh obrazov. Neyrokompyutery: razrabotka. primeneniye. 2021. T. 23. № 1. S. 17–31. (in Russian).
12. Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., Rabinovich A. Going deeper with convolutions. CoRR, abs/1409.4842. 2014.
13. Long J., Shelhamer E., Darrell T. Fully convolutional networks for semantic segmentation. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR). 2015. P. 3431–3440.
14. Ren S., He K., Girshick R., Sun J. Faster R-CNN: Towards realtime object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence. Jun. 2017. V. 39. № 6. P. 1137–1149.
15. Chang Liu, Xudong Jiang, Fellow, IEEE, Henghui Ding. Instance-Specific Feature Propagation for Referring Segmentation. arXiv:2204.12109v1. 26 Apr 2022.
16. Rizgar R. Zebari, Adnan Mohsin Abdulazeez, Diyar Qader Zeebaree, Dilovan Asaad Zebari, Jwan Najeeb Saeed A Comprehensive Review of Dimensionality Reduction Techniques for Feature Selection and Feature Extraction. Journal of Applied Science and Technology Trends. 2020. V. 1. № 2. P. 56–70.
17. Christian S., Sergey I., Vincent V., Alex A. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. arXiv:1602.07261v2. 2016.
18. Gao H., Zhuang L., Laurens van der Maaten, Kilian Q. Densely Connected Convolutional Networks. arXiv:1608.06993v5. 2018.
19. Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton ImageNet Classification with Deep Convolutional Neural Networks. Communications of the ACM. 2017. V. 60. Iss. 6. P. 84–90.
20. Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556v6. 2015.
21. Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., Rabinovich A. Going deeper with convolutions. CoRR. abs/1409.4842. 2014.
22. Lin T.Y., Maire M., Belongie S., Hays J., Perona P., Ramanan D., Doll´ar P., Zitnick C.L. Microsoft coco: Common objects in context. European Conference on Computer Vision. 2014. P. 740–755.
23. Kingma D.P., Ba J. Adam: A method for stochastic optimization. arXiv :1412.6980. 2014.
24. Phan T.H., Yamamoto K. Resolving Class imbalance in object detection with weighted cross entropy losses. arXiv:2006.01413. 2020.
25. Cheng B., Girshick R., Dollar P., Berg A., Kirillov A. Facebook AI Research (FAIR) Boundary IoU: Improving Object-Centric Image Segmentation Evaluation. arXiv:2103.16562v1. 2021.
26. He K., Gkioxari G., Dollar P., Girshick R. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. 2017. P. 2961–2969.
27. Azad R., Khosravi N., Merhof D. SMU-Net: Style matching U-Net for brain tumor segmentation with missing modalities. Institute of Imaging and Computer Vision. RWTH Aachen University. Germany. arXiv:2204.02961v1. 2022.