M.M. Grekov1, A.A. Sychugov2, E.I. Minakov3, S.F. Boev4

1,2,3 Tula State University (Tula, Russia)

4 PJSC AFK “Sistema” (Moscow, Russia)

1 grekov.web@yandex.ru, 2 xru2003@list.ru, 3 eminakov@bk.ru, 4 sfboev@gmail.com

Problem Statement. Automated penetration testing requires the formation of a sequence of attack actions that takes into account the current state of the network environment, the technical feasibility of the operations, and the logic of the penetration stages. Policies trained on a single configuration may depend on a specific route to the target, while direct behavioral cloning reproduces the expert's choice but does not generate a separate criterion for assessing the consequences of an action.

Goal. To increase the portability of attack strategies across different network environment configurations and improve their structural ordering in accordance with the stages of the penetration process while maintaining effectiveness and ensuring the feasibility of actions.

Results. A method is proposed that combines a dynamic mask of permissible actions, a phase attack model, an a priori expert policy, the reconstruction of the transition evaluation function along expert trajectories, and training on a family of network configurations. The mask reduced the proportion of invalid operations from 1.0 to 0. Incorporating the phase model increased the proportion of valid transitions between stages from 0.768 to 1.0, with similar effectiveness. Training on variable configurations increased the proportion of successful episodes on the validation set from 0.872 to 0.960. The reconstructed transition score increased the proportion of cases in which the expert transition occupied the first position from 0.552 to 0.779, and the average reverse position from 0.752 to 0.887.

Practical significance The method can be used in automated penetration testing systems to eliminate unfeasible operations, generate action sequences for attack stages, and verify the transfer of the strategy to modified network configurations.

The work was carried out with the financial support of a Grant in the form of a research grant from the Committee on Science and Innovation of the Tula Region No. 15 dated June 21, 2024.

Grekov M.M., Sychugov A.A., Minakov E.I., Boev S.F. Method for generating attack strategies for automated penetration testing based on recovered transition evaluation // Radioengineering. 2026. V. 90. № 9. P. 150−158. DOI: https://doi.org/10.18127/j00338486-202609-16

1. Myasnikov A.V. Primenenie tekhnologij mashinnogo obucheniya dlya optimizacii processa testirovaniya na proniknovenie. Problemy informacionnoj bezopasnosti. Komp'yuternye sistemy. 2019. № 2. S. 9–15 (in Russian).
2. Myasnikov A.V. Primenenie mashinnogo obucheniya s podkrepleniem v zadache testirovaniya na proniknovenie. Sovremennaya nauka: aktual'nye problemy teorii i praktiki. Ser. Estestvennye i tekhnicheskie nauki. 2020. № 11. S. 104–107. DOI: 10.37882/2223-2966.2020.11.26 (in Russian).
3. Grekov M.M. Model' avtomatizirovannoj sistemy imitacii atak na osnove mashinnogo obucheniya. Sistemy i sredstva informatiki. 2025. T. 35. № 3. S. 105–116. DOI: 10.14357/08696527250307 (in Russian).
4. Simon R., Mees W. SoK: a comparison of autonomous penetration testing agents. ARES’24: Proceedings of the 19th International Conference on Availability, Reliability and Security. New York: Association for Computing Machinery. 2024. Article 41. P. 1–10. DOI: 10.1145/3664476.3664484.
5. Sarraute C., Buffet O., Hoffmann J. POMDPs make better hackers: accounting for uncertainty in penetration testing. Proceedings of the AAAI Conference on Artificial Intelligence. 2012. V. 26. № 1. P. 1816–1824. DOI: 10.1609/aaai.v26i1.8363.
6. Kotenko I.V., Chechulin A.A. Primenenie grafov atak dlya ocenki zashchishchyonnosti komp'yuternyh setej i analiza sobytij bezopasnosti. Sistemy vysokoj dostupnosti. 2013. № 3. S. 103–110 (in Russian).
7. Ho J., Ermon S. Generative adversarial imitation learning. Advances in Neural Information Processing Systems. 2016. V. 29. P. 4565–4573.
8. Chen J., Hu S., Zheng H., Xing C., Zhang G. G AIL-PT: an intelligent penetration testing framework with generative adversarial imitation learning. Computers & Security. 2023. V. 126. Article 103055. DOI: 10.1016/j.cose.2022.103055.
9. Fu J., Luo K., Levine S. Learning robust rewards with adversarial inverse reinforcement learning. Proceedings of the 6th International Conference on Learning Representations. Vancouver. 2018.
10. Huang S., Ontanon S. A closer look at invalid action masking in policy gradient algorithms. The International FLAIRS Conference Proceedings. 2022. V. 35. DOI: 10.32473/flairs.v35i.130584.
11. Tobin J., Fong R., Ray A., Schneider J., Zaremba W., Abbeel P. Domain randomization for transferring deep neural networks from simulation to the real world. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2017. P. 23–30. DOI: 10.1109/IROS.2017.8202133.
12. Schulman J., Wolski F., Dhariwal P., Radford A., Klimov O. Proximal policy optimization algorithms. arXiv. 2017. arXiv:1707.06347. DOI: 10.48550/arXiv.1707.06347.