Integration of artificial intelligence into the process of developing test materials in discrete mathematics

E.V. Muzyukova

doi:10.18127/j19997493-202605-16

Integration of artificial intelligence into the process of developing test materials in discrete mathematics

research article UDC 378.146:378.147:004.8 Pages: 119-129 doi.org/10.18127/j19997493-202605-16

¹ Financial University under the Government of the Russian Federation (Moscow, Russia)

Keywords: Control and measurement materials (CMM)artificial intelligence (AI)generation of learning tasks discrete mathematics hybrid approach generative language models adaptive learning

Abstract

Problem statement. The digitalization of higher education and the widespread transition to competency-based knowledge assessment have significantly increased the workload of teachers associated with the systematic updating of task banks, revealing the limitations of traditional methods for creating control and measurement materials (CMMs).

Goal. To develop and theoretically substantiate a methodology for creating test items in the discipline of Discrete Mathematics based on the integration of parameterized templates and generative language models, which ensures didactic validity, mathematical correctness, and variability of test items.

Results. The didactic and content-mathematical requirements for CMM in discrete mathematics are substantiated; a formal assignment model is developed that takes into account the didactic goal, mathematical structure and generation parameters; a six-module system architecture is proposed; a multi-criteria scale for evaluating the quality of CMM is introduced; an expert approbation of CMM by three teachers is carried out (the average score of the hybrid strategy is 4.48 out of 5, against 3.96 and 3.50 for generative and template); a pilot pedagogical experiment was conducted with the participation of 24 students (Mann–Whitney criterion: U = 37.0, p = 0.044), which confirmed the statistically significant advantage of the hybrid approach.

Practical significance. The results can be applied in the design of automated knowledge control systems, the development of assessment tools, and the organization of adaptive testing.

Received: 17.09.2026
Approved after review: 29.09.2026
Accepted: 01.10.2026

How to cite

Музюкова Е.В. Интеграция искусственного интеллекта в процесс разработки контрольно-измерительных материалов по дискретной математике // Динамика сложных систем - XXI век. 2026. Т. 20. № 5. С. 119−129. DOI: https://doi.org/10.18127/j19997493-202605-16

References

Vavilova D.D., Kasatkina E.V., Fajzullin R.V. Ocenka potenciala masshtabirovaniya instrumentov iskusstvennogo intellekta v vy`sshem obrazovanii: rossijskij i zarubezhny`j opy`t. Obrazovanie i nauka. 2025. T. 27. № 9. S. 128–157. (in Russian).
Dagaev A.E. Iskusstvenny`j intellekt v zadache generacii distraktorov dlya testovy`x zadanij. Modelirovanie, optimizaciya i informacionny`e texnologii. 2025. T. 13. № 2. (in Russian).
Pugach O.I., Starceva N.V. Primenenie generativnogo iskusstvennogo intellekta v metodicheskoj rabote prepodavatelya vuza: razrabotka kontrol`no-izmeritel`ny`x materialov. Russian Journal of Education and Psychology. 2024. T. 15. № 5SE. S. 198–218. (in Russian).
Koshkina E.A., Bordovskaya N.V., Gnedy`x D.S., Xromova M.A., Dem`yanchuk R.V., Isxakova M.P., Baly`shev P.A. Generativny`j iskusstvenny`j intellekt v vy`sshem obrazovanii: obzor teoreticheskix podxodov i praktik primeneniya. Vy`sshee obrazovanie v Rossii. 2025. T. 34. № 6. S. 36–57. (in Russian).
Okishev S.V. Problema sozdaniya i ispol`zovaniya generatorov i reshatelej matematicheskix zadach. Mir nauki. 2018. № 3. (in Russian).
Achiam J., Adler S. et al. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774. 2023.
Boye J., Moell B. Large Language Models and Mathematical Reasoning Failures. arXiv preprint arXiv:2502.11574. 2025.
Zhang L., Graf E. Mathematical Computation and Reasoning Errors by Large Language Models. arXiv preprint arXiv:2508.09932. 2025.
Romera-Paredes B., Barekatain M., Novikov A. et al. Mathematical discoveries from program search with large language models. Nature. 2024. V. 625. P. 468–475.
Mittal C., Kartik K., Mausam, Singla P. FCoReBench: Can Large Language Models Solve Challenging First-Order Combinatorial Reasoning Problems? arXiv preprint arXiv:2402.02611. 2024.
Krutenko E.V., Shtejnberg B.Ya. Nekotory`e prichiny` oshibok iskusstvennogo intellekta pri reshenii kombinatorny`x zadach. Nauchny`j servis v seti Internet: trudy` XXVII Vserossijskoj nauchnoj konferencii. M.: IPM im. M.V. Keldy`sha. 2025. (in Russian).
Solov`ev V.I. Cifrovaya transformaciya vy`sshego obrazovaniya: opy`t i perspektivy` primeneniya generativny`x modelej. Mir novoj e`konomiki. 2023. T. 17. № 3. S. 45–53. (in Russian).
Lipatova E.V. Ispol`zovanie generativny`x nejrosetej v proektirovanii fondov ocenochny`x sredstv po vy`sshej matematike. Vestnik Finansovogo universiteta. Gumanitarny`e nauki. 2024. T. 14. № 1. S. 78–84. (in Russian).
Brown T., Mann B., Ryder N. et al. Language models are few-shot learners. Advances in neural information processing systems. 2020. V. 33. P. 1877–1901.
OpenAI. Learning to Reason with LLMs. OpenAI Blog. 2024.
Cobbe K., Kosaraju V., Bavarian M. et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. 2021.
Hendrycks D., Burns C., Kadavath S. et al. Measuring mathematical problem solving with the MATH dataset. arXiv preprint arXiv:2103.03874. 2021.
Lightman H., Kosaraju V., Burda Y. et al. Lets verify step by step. arXiv preprint arXiv:2305.20050. 2023.
Garcez A. d., Lamb L.C. Neurosymbolic AI: The 3rd Wave. Artificial Intelligence. 2023. V. 321. Art. 103960.
Chaudhuri S., Gulwani S., Kincaid M. et al. Neurosymbolic Programming. Foundations and Trends in Programming Languages. 2021. V. 7. № 3. P. 158–243.
Symbolic Solvers and LLM Integration in Task Generation / ed. by A. Smith. Journal of Educational Computing Research. 2024. V. 62. № 2. P. 315–334.
Sweller J. Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction. 1994. V. 4. № 4. P. 295–312.
Mayer R.E. Multimedia learning. Psychology of Learning and Motivation. 2002. V. 41. P. 85–139.
Romanov A.M. Osobennosti prepodavaniya diskretnoj matematiki dlya studentov IT-napravlenij. Komp`yuterny`e instrumenty` v obrazovanii. 2023. № 2. S. 48–59. (in Russian).
Chely`shkova M.B. Teoriya i praktika konstruirovaniya testovy`x zadanij. M.: Logos. 2002. 432 s. (in Russian).
Anderson L.W., Krathwohl D.R. A taxonomy for learning, teaching, and assessing: A revision of Blooms taxonomy of educational objectives. New York: Longman, 2001.
Konnova L.P., Lipagina L.V., Olexova E.F., Ry`lov A.A., Stepanyan I.K. Korrektiruyushhij podxod k ocenivaniyu akademicheskix dostizhenij studentov v LMS Moodle. Informatika i obrazovanie. 2022. 37(6):75–85. DOI: 10.32517/0234-0453-2022-37-6-75-85 (in Russian).
Val`kova Yu.E. Ispol`zovanie texnologij iskusstvennogo intellekta dlya podgotovki i napisaniya nauchny`x statej. Informatika i obrazovanie. 2024;39(6):38–52. DOI: 10.32517/0234-0453-2024-39-6-38-52 (in Russian).
Strelkova I.A. «Prokrustovo lozhe» cifrovizacii. E`konomika. Nalogi. Pravo. 2022;15(4):77-85. DOI: 10.26794/1999-849X 2022-15-4-77-85 (in Russian).
Maksimova M.V., Frolova O.V., E`tuev X.X., Aleksandrova L.D. Adaptivnoe personalizirovannoe obuchenie: vnedrenie sovremenny`x texnologij v vy`sshem obrazovanii. Informatika i obrazovanie. 2023;38(4):14–27. DOI: 10.32517/0234-0453-2023-38-4-14-27 (in Russian).
Brovkina N.E., Goncharenko M.A. Iskusstvenny`j intellekt v kreditno-finansovoj sfere: vektor razvitiya, prognozy` i ocenka tekushhix problem. E`konomika. Nalogi. Pravo. 2026;19(2):31-39. DOI: 0.26794/1999-849X-2026-19-2-31-39 (in Russian).
Eliseeva E.N. Smeshanny`j format obucheniya – perspektivnoe napravlenie podgotovki studentov e`konomicheskix special`nostej. E`konomika. Nalogi. Pravo. 2022;15(6):95-103. DOI: 10.26794/1999-849X 2022-15-6-95-103 (in Russian).

[1] Vavilova D.D., Kasatkina E.V., Fajzullin R.V. Ocenka potenciala masshtabirovaniya instrumentov iskusstvennogo intellekta v vy`sshem obrazovanii: rossijskij i zarubezhny`j opy`t. Obrazovanie i nauka. 2025. T. 27. № 9. S. 128–157. (in Russian).

[2] Dagaev A.E. Iskusstvenny`j intellekt v zadache generacii distraktorov dlya testovy`x zadanij. Modelirovanie, optimizaciya i informacionny`e texnologii. 2025. T. 13. № 2. (in Russian).

[3] Pugach O.I., Starceva N.V. Primenenie generativnogo iskusstvennogo intellekta v metodicheskoj rabote prepodavatelya vuza: razrabotka kontrol`no-izmeritel`ny`x materialov. Russian Journal of Education and Psychology. 2024. T. 15. № 5SE. S. 198–218. (in Russian).

[4] Koshkina E.A., Bordovskaya N.V., Gnedy`x D.S., Xromova M.A., Dem`yanchuk R.V., Isxakova M.P., Baly`shev P.A. Generativny`j iskusstvenny`j intellekt v vy`sshem obrazovanii: obzor teoreticheskix podxodov i praktik primeneniya. Vy`sshee obrazovanie v Rossii. 2025. T. 34. № 6. S. 36–57. (in Russian).

[5] Okishev S.V. Problema sozdaniya i ispol`zovaniya generatorov i reshatelej matematicheskix zadach. Mir nauki. 2018. № 3. (in Russian).

[6] Achiam J., Adler S. et al. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774. 2023.

[7] Boye J., Moell B. Large Language Models and Mathematical Reasoning Failures. arXiv preprint arXiv:2502.11574. 2025.

[8] Zhang L., Graf E. Mathematical Computation and Reasoning Errors by Large Language Models. arXiv preprint arXiv:2508.09932. 2025.

[9] Romera-Paredes B., Barekatain M., Novikov A. et al. Mathematical discoveries from program search with large language models. Nature. 2024. V. 625. P. 468–475.

[10] Mittal C., Kartik K., Mausam, Singla P. FCoReBench: Can Large Language Models Solve Challenging First-Order Combinatorial Reasoning Problems? arXiv preprint arXiv:2402.02611. 2024.

[11] Krutenko E.V., Shtejnberg B.Ya. Nekotory`e prichiny` oshibok iskusstvennogo intellekta pri reshenii kombinatorny`x zadach. Nauchny`j servis v seti Internet: trudy` XXVII Vserossijskoj nauchnoj konferencii. M.: IPM im. M.V. Keldy`sha. 2025. (in Russian).

[12] Solov`ev V.I. Cifrovaya transformaciya vy`sshego obrazovaniya: opy`t i perspektivy` primeneniya generativny`x modelej. Mir novoj e`konomiki. 2023. T. 17. № 3. S. 45–53. (in Russian).

[13] Lipatova E.V. Ispol`zovanie generativny`x nejrosetej v proektirovanii fondov ocenochny`x sredstv po vy`sshej matematike. Vestnik Finansovogo universiteta. Gumanitarny`e nauki. 2024. T. 14. № 1. S. 78–84. (in Russian).

[14] Brown T., Mann B., Ryder N. et al. Language models are few-shot learners. Advances in neural information processing systems. 2020. V. 33. P. 1877–1901.

[15] OpenAI. Learning to Reason with LLMs. OpenAI Blog. 2024.

[16] Cobbe K., Kosaraju V., Bavarian M. et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. 2021.

[17] Hendrycks D., Burns C., Kadavath S. et al. Measuring mathematical problem solving with the MATH dataset. arXiv preprint arXiv:2103.03874. 2021.

[18] Lightman H., Kosaraju V., Burda Y. et al. Lets verify step by step. arXiv preprint arXiv:2305.20050. 2023.

[19] Garcez A. d., Lamb L.C. Neurosymbolic AI: The 3rd Wave. Artificial Intelligence. 2023. V. 321. Art. 103960.

[20] Chaudhuri S., Gulwani S., Kincaid M. et al. Neurosymbolic Programming. Foundations and Trends in Programming Languages. 2021. V. 7. № 3. P. 158–243.

[21] Symbolic Solvers and LLM Integration in Task Generation / ed. by A. Smith. Journal of Educational Computing Research. 2024. V. 62. № 2. P. 315–334.

[22] Sweller J. Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction. 1994. V. 4. № 4. P. 295–312.

[23] Mayer R.E. Multimedia learning. Psychology of Learning and Motivation. 2002. V. 41. P. 85–139.

[24] Romanov A.M. Osobennosti prepodavaniya diskretnoj matematiki dlya studentov IT-napravlenij. Komp`yuterny`e instrumenty` v obrazovanii. 2023. № 2. S. 48–59. (in Russian).

[25] Chely`shkova M.B. Teoriya i praktika konstruirovaniya testovy`x zadanij. M.: Logos. 2002. 432 s. (in Russian).

[26] Anderson L.W., Krathwohl D.R. A taxonomy for learning, teaching, and assessing: A revision of Blooms taxonomy of educational objectives. New York: Longman, 2001.

[27] Konnova L.P., Lipagina L.V., Olexova E.F., Ry`lov A.A., Stepanyan I.K. Korrektiruyushhij podxod k ocenivaniyu akademicheskix dostizhenij studentov v LMS Moodle. Informatika i obrazovanie. 2022. 37(6):75–85. DOI: 10.32517/0234-0453-2022-37-6-75-85 (in Russian).

[28] Val`kova Yu.E. Ispol`zovanie texnologij iskusstvennogo intellekta dlya podgotovki i napisaniya nauchny`x statej. Informatika i obrazovanie. 2024;39(6):38–52. DOI: 10.32517/0234-0453-2024-39-6-38-52 (in Russian).

[29] Strelkova I.A. «Prokrustovo lozhe» cifrovizacii. E`konomika. Nalogi. Pravo. 2022;15(4):77-85. DOI: 10.26794/1999-849X 2022-15-4-77-85 (in Russian).

[30] Maksimova M.V., Frolova O.V., E`tuev X.X., Aleksandrova L.D. Adaptivnoe personalizirovannoe obuchenie: vnedrenie sovremenny`x texnologij v vy`sshem obrazovanii. Informatika i obrazovanie. 2023;38(4):14–27. DOI: 10.32517/0234-0453-2023-38-4-14-27 (in Russian).

[31] Brovkina N.E., Goncharenko M.A. Iskusstvenny`j intellekt v kreditno-finansovoj sfere: vektor razvitiya, prognozy` i ocenka tekushhix problem. E`konomika. Nalogi. Pravo. 2026;19(2):31-39. DOI: 0.26794/1999-849X-2026-19-2-31-39 (in Russian).

[32] Eliseeva E.N. Smeshanny`j format obucheniya – perspektivnoe napravlenie podgotovki studentov e`konomicheskix special`nostej. E`konomika. Nalogi. Pravo. 2022;15(6):95-103. DOI: 10.26794/1999-849X 2022-15-6-95-103 (in Russian).

Abstract

How to cite

Related articles