L.I. Averina1, N. E. Guterman2, D.Yu. Charkin3

1-3 JSC «Concern «Sozvezdie» (Voronezh, Russia)

1 averina@phys.vsu.ru; 2 n.guterman@internet.ru; 3 d.yu.charkin@sozvezdie.su

Multichannel devices with adaptive beamforming are beginning to play an increasingly important role in microwave wireless communication systems. To reduce the circuit and technological complexity of their implementation, beamforming is increasingly carried out in the analog domain using phase shifters. However, this significantly complicates the process of finding the optimal solution for weight coefficients, generating an NP-hard optimization problem. This paper proposes a neural network approach based on deep reinforcement learning and advantage actor-critic algorithm for solving the problem of adaptive analog beamforming of microwave multichannel systems. The proposed approach involves the use of neural networks to approximate the agent's strategy and allows eliminating channel estimation. The conducted simulation modeling showed the effectiveness of the proposed solutions, demonstrating the gain of the antenna array close to optimal both in the absence of interference sources and in their presence.

Averina L.I., Guterman N.E., Charkin D.Yu. Neural network based approach for adaptive analog beamforming on receiver side in microwave multichannel systems. Radiotekhnika. 2025. V. 89. № 9. P. 133−144. DOI: https://doi.org/10.18127/j00338486-202509-14 (In Russian)

1. Askarov S., Kizilirmak R.C., Ukaegbu I.A. Path loss analysis and system coverage for 28 GHz, 60 GHz and 73 GHz band wireless communication systems. Proc. 2023 Int. Conf. on Application of Information and Communication Technologies (Baku. Azerbaijan). N.Y.: IEEE. 2023.
2. Zhang W., Xia X., Fu Y., Bao X. Hybrid and full-digital beamforming in mm wave massive MIMO systems: a comparison considering low-resolution ADCs. China Communications. June 2019. V. 16. № 6. Р. 91.
3. Shehata M., Mokh A., Crussiere M., et al. On the equivalence between hybrid and full digital beamforming in mm wave communications. Proc. 2019 26th Int. Conf. on Telecommunications (Hanoi. Vietnam). N.Y.: IEEE. 2019. Р. 391.
4. Sohrabi F., Yu W. Hybrid digital and analog beamforming design for large-scale MIMO systems. 2015 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (South Brisbane. QLD. Australia). N.Y.: IEEE. 2015. Р. 2929.
5. Ahmed I., Khammari H., Shahid A. A survey on hybrid beamforming techniques in 5G: architecture and system model perspectives. IEEE Communications Surveys & Tutorials. 2018. V. 20. № 4. Р. 3060.
6. Mabrouki S., Dayoub I., Li Q., Berbineau M. Codebook designs for millimeter-wave communication systems in both low- and high-mobility: achievements and challenges. IEEE Access. 2022. V. 10. Р. 25786.
7. Eldar Y.C., Goldsmith A., Gunduz D., Poor H.V. Machine learning and wireless communications. Cambridge: Cambridge University Press. 2022.
8. Luo F.-L. Machine learning for future wireless communications. N.-Y.: Wiley IEEE-Press. 2020.
9. Luong N.C., Hoang D.T., Gong S., et al. Applications of deep reinforcement learning in communications and networking: a survey. IEEE Communications Surveys & Tutorials. 2019. V. 21. № 4. Р. 3133.
10. Mao Q., Hu F., Hao Q. Deep learning for intelligent wireless networks: a comprehensive survey. IEEE Communications Surveys & Tutorials. 2018. V. 20. № 4. Р. 2595.
11. El Halawany B.M., Hashima S., Hatano K., et al. Leveraging machine learning for millimeter wave beamforming in beyond 5G networks. IEEE Systems Journal. 2022. V. 16. № 2. Р. 1739.
12. Wu W., Cheng N., Zhang N., et al. Fast mmwave beam alignment via correlated bandit learning. IEEE Transactions on Wireless Communications. 2019. V. 18. № 12. Р. 5894.
13. Zhang J., Huang Y., Wang J., et al. Intelligent interactive beam training for millimeter wave communications. IEEE Transactions on Wireless Communications. 2021. V. 20. № 3. Р. 2034-2048.
14. Chu M., Liu A., Lau V.K.N., et al. Deep reinforcement learning based end-to-end multiuser channel prediction and beamforming. IEEE Transactions on Wireless Communications. 2022. V. 21. № 12. Р. 10271-10285.
15. Lizarraga E.M., Maggio G.N., Dowhuszko A.A. Hybrid beamforming algorithm using reinforcement learning for millimeter wave wireless systems. 2019 XVIII Workshop on Information Processing and Control (Salvador, Brazil). N.-Y.: IEEE. 2019. Р. 253.
16. Le Magoarou L., Yassine T., Paquelet S., Crussiere M. Deep learning for location based beamforming with Nlos channels. 2022 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (Singapore. Singapore). N.-Y.: IEEE. 2022. Р. 8812.
17. Wang L., Ai B., Niu Y., et al. Adaptive beam alignment based on deep reinforcement learning for high speed railways. 2022 IEEE 95th Vehicular Technology Conf. (Helsinki. Finland). N.-Y.: IEEE. 2022. Р. 1.
18. Dmitriyev E., Rogozhnikov E., Duplishcheva N., Novichkov S. Calculating beamforming vectors for 5G system applications. Symmetry. 2021. V. 13. № 12. Р. 2423.
19. Love D.J., Heath R.W. Equal gain transmission in multiple-input multiple-output wireless systems. IEEE Transactions on Com-munications. 2003. V. 51. № 7. Р. 1102-1110.
20. Liu X., Zhang J., Tang P., et al. Channel sparsity variation and model-based analysis on 6, 26, and 132 GHz measurements. https://arxiv.org/abs/2302.08772.
21. Gao Z., Hu C., Dai L., Wang Z. Channel estimation for millimeter-wave massive MIMO with hybrid precoding over frequency-selective fading channels. IEEE Communications Letters. 2016. V. 20. № 6. Р. 1259-1262.
22. Zhang Y., Alrabeiah M., Alkhateeb A. Reinforcement learning for beam pattern design in millimeter wave and massive MIMO systems. 2020 54th Asilomar Conf. on Signals, Systems and Computers (Pacific Grove. CA., USA). N.-Y.: IEEE. 2020. Р. 445.
23. Zhang Y., Alkhateeb A. Online beam learning for interference nulling in hardware-constrained mm wave MIMO systems. 2022 56th Asilomar Conf. on Signals, Systems and Computers (Pacific Grove. CA. USA). N.-Y.: IEEE. 2022. Р. 200.
24. Graesser L., Keng W.L. Foundations of deep reinforcement learning. Theory and Practice in Python. Boston: Addison-Wesley Prof. 2019.
25. Sutton R.S., Barto A.G. Reinforcement learning: an introduction. Boston: MIT Press. 2018.