350 rub
Journal Highly available systems №4 for 2014 г.
Article in number:
Construction of hardware-software complexes architecture to improve massively data processing
Keywords: big data processing parallel data processing massively data processing hardware-software complexes architecture data access methods
Authors:
V. I. Munerman - Ph.D. (Eng.), Associate Professor, Department of Phisics & Math, Smolensk State University. E-mail: vimoon@gmail.com
Abstract:
The paper discusses the problem of optimizing the mass processing of large volumes of data, which refers to the area of study called BIG DATA. It is shown that the current state of computer technology and the possibilities of programming systems allow optimization at different levels and by different methods. The technology development of data processing based on models linking common data models and model calculations is considered. The definition of massively data processing is given, and its characteristics are defined. The basic directions of efficiency are defined, notably the development of data models that provide the greatest consistency with existing data models and architectures of computer systems, the development of models, based on these ways of organizing data in external memory and implementation of operations algorithms in the most appropriate data structures. Operations of massively data processing and their properties are defined. The properties of binding models are defined, particularly matching data models and calculation models, the procedurality, the parallelism of algebraic operations, the ability to queries optimization, and the object-oriented paradigm. The descriptions of the set-theoretic and multidimensional matrix models are considered. It is shown that on the basis of them it is possible to use two approaches to parallel implementation of the massively data processing objectives: in the data-flow model of computation for computing architecture with associative distribution of resources, and in the matrix model of computation with the generalization of parallel algorithms that implement operations on two-dimensional matrices to multidimensional matrices. An example illustrating the implementation of the proposed methods on the database machine is considered. The paper also proposes a comprehensive training course designed for teaching students and advanced training of specialists in the considered area of data processing.
Pages: 3-16
References
  1. O-Leary, Daniel L. Enterprise resource planning systems. Cambridge University Press. 2000. 232 s.
  2. Siniczy'n I.N., Shalamov A.S. Lekczii po teorii sistem integrirovannoj logisticheskoj podderzhki. M.: TORUS PRESS. 2012. 624 s.
  3. Voevodin V.V. Vy'chislitel'naya matematika i struktura algoritmov. M.: Izd. MGU. 2006. 112 s.
  4. Munerman V.I. Ob''ektno-orientirovannaya model' massovoj obrabotki danny'x // Sistemy' vy'sokoj dostupnosti. 2011. T. 7.
    № 4. S. 72 - 74.
  5. Levin N.A., Munerman V.I. Realizacziya ob''ektno-orientirovannoj modeli massovoj obrabotki danny'x // Sistemy' vy'sokoj dostupnosti. 2012. T. 8. № 3. S. 23 - 25.
  6. Munerman V.I. Mnogomerno-matrichnaya model' massovoj obrabotki danny'x // Sistemy' vy'sokoj dostupnosti. 2012. T. 8.
    № 3. S. 19 - 22.
  7. Dejt, K. Dzh. Vvedenie v sistemy' baz danny'x. Izd. 8-e: Per. s angl. M.: Izdatel'skij dom «Vil'yame». 2005. 1328 s.
  8. Makarov D.I., Munerman V.I. Parallel'naya realizaczii operaczii soedineniya dlya massovoj obrabotki danny'x // Sistemy' vy'sokoj dostupnosti. 2012. T. 8. № 3. S. 26 - 28.
  9. Levin N.A., Munerman V.I. Modeli obrabotki bol'shix ob''emov danny'x v sistemax massovogo parallelizma // Sistemy' vy'sokoj dostupnosti. 2013. T. 9. № 1. S. 35 - 43.
  10. Munerman V.I. Realizacziya obrabotki bol'shix ob''emov danny'x na simmetrichny'x mul'tiproczessorny'x sistemax // Sistemy' vy'sokoj dostupnosti. 2013. T. 9. № 2. S. 36 - 39.
  11. Kuzneczov S.D. Osnovy' baz danny'x. M.: Internet universitet informaczionny'x texnologij; Binom. Laboratoriya znanij. 2007. 484 s.
  12. Si Tu Tant Sin Parallel'ny'e i posledovatel'ny'e varianty' dinamicheskogo programmirovaniya / Sovremenny'e informaczionny'e texnologii i IT-obrazovanie / Sb. izbranny'x trudov VII Mezhdunar. nauchno-praktich. konf. // Pod red. prof. V.A. Suxomlina. M.: INTUIT.RU. 2012. S. 932 - 939.
  13. Emel'chenkov E.P., Munerman V.I., Siniczy'n I.N. Opy't spol'zovaniya sistem komp'yuternoj matematiki v upravlenii proczessom obucheniya matematicheskim discziplinam / Sovremenny'e informaczionny'e texnologii i IT-obrazovanie / Sb. izbranny'x trudov VII Mezhdunar. nauchno-praktich. konf. // Pod red. prof. V.A. Suxomlina. M.: INTUIT.RU. 2012. S. 673 - 680.
  14. http://www.redbooks.ibm.com/redbooks/pdfs/redp4725-00-ru.pdf
  15. Munerman V.I. Opy't massovoj obrabotki danny'x v oblachny'x sistemax (na primere Windows Azure) // Sistemy' vy'sokoj dostupnosti. 2014. T. 9. № 2. S. 3 - 8.