A.Y. Virasova1, D.I. Klimov2, O.E. Khromov3, I.R. Gubaidullin4, V.V. Oreshko5

1−5 JSC "Russian Space Systems" FKA "Roscosmos" (Moscow, Russia)

Formulation of the problem. Over the past few years, there has been little progress in object detection techniques. The most efficient are complex computational ensemble methods, which usually combine several low-level image properties with high-level properties. However, every day interest in artificial intelligence is growing, and the idea of using neural networks on board a spacecraft, with the possibility of making decisions and issuing one-time commands, is very promising, since it makes it possible to analyze a large data stream in real time without resorting to ground station, thereby not losing information when transmitting a packet.

The purpose of the work is to conduct research on the possibility of effective use of modern models of neural networks, to develop a methodology for their use in the problem of object detection and analysis of the element base for hardware implementation with the possibility of using convolutional neural networks for thermovideotelemetry on board a spacecraft.

Results of work. An approach has been formulated that combines two key ideas: 1) application of high-throughput convolutional neural networks for downward processing of image regions in order to localize and segment objects; 2) preliminary training for the auxiliary task, followed by fine tuning of the domain, which gives a significant increase in performance in the case when the training data is insufficient. The analysis of the element base for the possibility of hardware implementation of neural networks on board a spacecraft using electrical radio products of domestic and foreign production is carried out.

Practical significance. The efficiency of preliminary network training for an auxiliary task is shown, followed by fine tuning of the subject area. A technique is described that makes it possible to increase the average accuracy of detecting objects in an image by more than 30%. The analysis of the existing element base, the possibility of hardware implementation of neural networks on board the spacecraft using electrical radio products of domestic and foreign production, as well as the criteria for selecting key elements.

Virasova A.Y., Klimov D.I., Khromov O.E., Gubaidullin I.R., Oreshko V.V. Rich feature hierarchies for accurate object detection  and semantic segmentation. Radiotekhnika. 2021. V. 85. № 9. P. 115−126. DOI: https://doi.org/10.18127/j00338486-202109-11 (In Russian)

1. Girshik Ross Dzheff, Darrell Donah'ju Trevor, Malik Dzhitendra. Kalifornijskij universitet v Berkli: Bogatye ierarhii funkcij dlja tochnogo obnaruzhenija ob#ektov i semanticheskoj segmentacii. 2014 (in Russian).
2. Lou D. Otlichitel'nye osobennosti izobrazhenija ot masshtabno-invariantnyh kljuchevyh tochek. IJCV. 2004 (in Russian).
3. Dalal N., Triggs B. Gistogrammy orientirovannyh gradientov dlja obnaruzhenija cheloveka. CVPR. 2005 (in Russian).
4. Fukusima K. Neokognitron: model' samoorganizujushhejsja nejronnoj seti dlja mehanizma raspoznavanija obrazov, na kotoryj ne vlijaet izmenenie polozhenija. Biologicheskaja kibernetika. 1980. T. 36. № 4. S. 193–202 (in Russian).
5. Le Kun Ju., Bottu L., Benzhio Ju., Haffner P. Primenenie gradientnogo obuchenija dlja raspoznavanija dokumentov. Proc. IEEE. 1998 (in Russian).
6. Krizhevskij A., Suckever I., Hinton G. Klassifikacija Image Net s glubokimi svertochnymi nejronnymi setjami. NIPS. 2012 (in Russian).
7. Djen Dzh., Berg A., Satish S., Su H., Hosla A., Fej-Fej L. Konkurs po krupnomasshtabnomu vizual'nomu raspoznavaniju Image Net. 2012 (ILSVRC2012). http://www.image-net.org/ issues/LSVRC/2012/ (in Russian).
8. Deng J., Dong W., Socher R., Li L.-J., Li K., Fej-Fjej L. Image Net: krupnomasshtabnaja baza dannyh ierarhicheskih izobrazhenij. CVPR. 2009 (in Russian).
9. Segedi Ch., Toshev A., Jerhan D. Glubokie nejronnye seti dlja obnaruzhenija ob’ektov. NIPS. 2013 (in Russian).
10. Rouli H.A., Baludzha S., Kanade T. Obnaruzhenie lic na osnove nejronnoj seti. TPAMI. 1998 (in Russian).
11. Vaillant R., Monrocq C., Le Cun Y. Original'nyj podhod k lokalizacii ob’ektov na izobrazhenijah. IEE Proc. on Vision, Image and Signal Processing. 1994 (in Russian).
12. Gu K., Lim Dzh.Dzh., Arbelez P., Malik Dzh. Raspoznavanie po regionam. CVPR. 2009 (in Russian).
13. Uidzhlings Dzh., van de Sande K., Gevers T., Smelders A. Vyborochnyj poisk dlja raspoznavanija ob’ektov. IJCV. 2013 (in Russian).
14. Karrejra Dzh., Sminchiesku K. CPMC: avtomaticheskaja segmentacija ob#ektov s ispol'zovaniem parametricheskih ogranichennyh razrezov. TPA (in Russian).
15. Sermanet P., Kavukchuoglu K., Chintala S., Le Kun Ju. Obnaruzhenie peshehodov s nekontroliruemym mnogostupenchatym obucheniem funkcij. CVPR. 2013 (in Russian).
16. Fel'censhvalb P., Girshik R., Makallester D., Ramanan D. Obnaruzhenie ob#ektov s pomoshh'ju detektivno-obuchennyh modelej na osnove detalej. TPAMI, 2010 (in Russian).
17. Girshik R., Fel'censhvalb P., Makallester D. Diskriminacionno obuchennye modeli deformiruemyh detalej. Вып. 5. http://www.cs. berkeley.edu/˜rbg/latent-v5/ (in Russian).
18. Aleksej B., Deseljers T., Ferrari V. Izmerenie ob#ektnosti okon izobrazhenij. TPAMI. 2012 (in Russian).
19. Jendres I., Hojem D. Predlozhenija po nezavisimym ot kategorii ob’ektam. ECCV. 2010 (in Russian).
20. Arbelaes P., Pont-Tuset Dzh., Bjerron Dzh., Markes F., Malik Dzh. Mnogomasshtabnaja kombinatornaja gruppirovka. CVPR. 2014 (in  Russian).
21. Czja Ju. Caffe: svertochnaja arhitektura s otkrytym ishodnym kodom dlja bystrogo vstraivanija funkcij. 2013. http://caffe.berkeleyvision.org/ (in Russian).
22. Virjasova A.Ju., Gladkih A.A. Analiz predmetnoj oblasti i razrabotka bazy dannyh dlja realizacii defektoskopii integral'nyh struktur. Sb. trudov Mezhdunar. molodezhnoj nauch.-tehnich. konf. «Naukoemkie tehnologii i intellektual'nye sistemy. 2018 (in Russian).
23. Klimov D.I. Videotelemetricheskij kontrol' promyshlennyh izdelij. M.: Fizmatlit. 2018. T. 5. Vyp. 2. S. 89-98 (in Russian).
24. Virjasova A.Ju., Ishin I.A. Analiz predmetnoj oblasti i razrabotka bazy dannyh dlja realizacii defektoskopii integral'nyh struktur. Academy. 2017. № 12(27). S. 26-29 (in Russian).
25. Gribachev V. Jelementnaja baza apparatnyh realizacij nejronnyh setej. Komponenty i tehnologii. 2006. № 8 (in Russian).