A.A. Bezgodov, K.V. Knyazkov, S.V. Kovalchuk, A.V. Boukhanovsky
The article presents new approach for building interactive virtual reality systems. The approach is based on the composition of three key technologies: virtual reality, brain-computer interfaces (BCI), and cloud computing. The main goal is building of the instrumental toolbox for solving contemporary e-Science tasks, which often require analysis of large and complex data. The approach is based on common semantic descriptive union of simulation data, visual 3d-scene within virtual reality environment and cognitive image within the user’s psyche. The basis for the semantic union is formed by Virtual Simulation Objects (VSO) concept, which defines principles of description and manipulation for semantic objects representing some «real-world» objects and allowing simulation running. The relation between objects within virtual reality, operator’s activities and VSO-based semantic description is proposed. BCI cycle is described with respect to the semantic description and technologies being used.
The key feature of proposed approach is analysis of emotion component within brain activity of the experts to identify subjective estimation (including nonverbal and unconscious). Analysis of user’s attention focus combined with measurement of brain activity allows to map this emotional estimation onto virtual reality space and semantic description of involved objects. So the «map of emotional estimation» can be defined and used for the purpose of automatic user support during virtual reality experience.
The technology for interactive virtual reality environments building proposed. The technology is based on CLAVIRE (CLoud Application VIRtual Environment) platform which allows to use cloud computing for solving wide range of tasks. The architecture of interactive system for BCI-based virtual reality systems implementation is developed using the CLAVIRE. It uses virtual test bed concept to form the environment for interactive 3d-scenes implementation. Interactive Workflow technology is used for dynamic user-system interaction support. This technology is supported by CLAVIRE. It allows to compose all the services within single composite application with run-time support of theirs interaction.