A.G. Korobeynikov1

1 SPbF IZMIRAN (Saint-Petersburg, Russia)

1 Korobeynikov_A_G@mail.ru

This paper presents an approach based on the methods of the rapidly developing BIG DATA technology, which makes it possible to preprocess large volume data obtained, for example, from geomagnetic observatories that are part of the widely branched international INTERMAGNET network. This data is taken at a rate of once per second (1 Hz). Thus, the data accumulated over several years is very large. Therefore, working with them requires the use of modern information technologies, such as BIG DATA technology. The use of modern BIG DATA technologies for working with big data allows solving a fairly wide class of geophysical problems. It follows that the development and / or further development of methods for processing big data in relation to the problems of geophysics and based on the use of modern BIG DATA technology is an urgent problem. Based on this, the purpose of this work is to develop, based on BIG DATA technology, an approach to preprocessing large volumes of data in the ULF range and provide an example of the practical implementation of this approach in the MATLAB system. To test the approach implemented in MATLAB, we used data obtained from the INTERMAGNET official website, transmitted by the Lycksele geomagnetic observatory (Sweden, Geological Survey of Sweden, international IAGA code LYC). The choice of the MATLAB system is due to the presence in it of well-developed and tested reliable tools that implement a fairly representative family of BIG DATA methods. This circumstance allows to significantly reduce the development time with a sufficiently high quality and reliability of software tools that allow you to work with large data volumes. The practical significance lies in the reduction of the initial data preprocessing time. The data obtained as a result of preprocessing can later be used in solving various problems. For example, for a detailed analysis of: the properties of disturbances of the Earth's magnetic field in the range of long-period pulsations; the nature of the interaction of waves and particles in the magnetosphere; movement of the magnetic poles.

The paper presents in graphical form the original big data received from the observatory by the Lycksele observatory for the period from January 1, 2018 to December 31, 2022, as well as the results of their preprocessing.

Korobeynikov A.G. BIG DATA methods for preprocessing data ULF-range received from geomagnetic observatories included in the intermagnet international network. Radiotekhnika. 2023. V. 87. № 12. P. 171−176. DOI: https://doi.org/10.18127/j00338486-202312-18 (In Russian)

1. Korobeynikov A.G., Grishentsev A.Y., Velichko E.N., Aleksanin S.A., Fedosovskii M.E., Bondarenko I.B., Korikov C.C. Calculation of regularization parameter in the problem of blur removal in digital image. Optical Memory & Neural Networks (Information Optics). 2016. V. 25. № 3. P. 184-191.
2. D'jakonov V.P. MATLAB i SIMULINK dlja radioinzhenerov. M.: DMK Press. 2016. 976 s. (in Russian).
3. Novgorodcev A.B. Raschet jelektricheskih cepej v MATLAB: Uchebnyj kurs. SPb: Piter. 2004. 250 s. (in Russian).
4. Matjushkin I.V. Modelirovanie i vizualizacija sredstvami MATLAB fiziki nanostruktur. M.: Tehnosfera. 2011. 168 s. (in Russian).
5. Korobeynikov A.G., Fedosovsky M.E., Zharinov I.O., Shukalov A.V., Gurjanov A.V. Development of conceptual modeling method to solve the tasks of computer-aided design of difficult technical complexes on the basis of category theory. International Journal of Applied Engineering Research. 2017. V. 12. № 6. Р. 1114-1122.
6. Gajduk A.R., Beljaev V.E., P’javchenko T.A. Teorija avtomaticheskogo upravlenija v primerah i zadachah s reshenijami v MATLAB: Ucheb. posobie. Izd. 3-e, ster. SPb: Izdatel'stvo «Lan'». 2016. 464 s. (in Russian).
7. Porshnev S.V. Komp'juternoe modelirovanie fizicheskih processov v pakete MATLAB: Ucheb. posobie. Izd. 2-e, ispr. SPb: Izdatel'stvo «Lan'». 2011. 736 s. (in Russian).
8. Frisk V.V., Ganin V.I., Stepanova A.G. Komp'juternyj analiz i modelirovanie jelektricheskih cepej postojannogo toka v srede MATLAB: Ucheb.-metodich. posobie. M.: SOLON-PRESS. 2021. 32 s. (in Russian).
9. Korobejnikov A.G. Primenenie iskusstvennyh nejronnyh setej v sistemah avtomaticheskogo upravlenija magnitnoj levitaciej. Programmnye produkty i sistemy. 2022. T. 35. № 3. S.452–457. DOI: 10.15827/0236-235X.139.452-457 (in Russian).
10. Tereshhenko E.D., Tereshhenko P.E. Kvazistacionarnoe priblizhenie v zadache vozbuzhdenija nizkochastotnyh jelektro-magnitnyh polej v litosfere. Zhurnal tehnicheskoj fiziki. 2021. T. 91. № 1. S. 82-88 (in Russian).
11. Makshanov A.V., Zhuravlev A.E., Tyndykar' L.N. Bol'shie dannye. Big Data. Izd. 2-e, ster. SPb: Izdatel'stvo «Lan'». 2022. 188 s. (in Russian).
12. Bestugin A.R., Antokhin E.A., Dvornikov S.V., Dvornikov S.S., Kirshina I.A. Detection of discontinuous signals from correlation processing of input realisations. Radiotekhnika. 2023. V. 87. № 6. P. 5−11. DOI: https://doi.org/10.18127/j00338486-202306-01 (in Russian).
13. Korobeynikov A. G. Processing and analysis of data from the Russian segment of the world network of magnetic observatory INTERMAGNET. International Journal of Humanities and Natural Sciences. 2018. № 8. P. 91-98 (in Russian).