A.D. Voronkov1, S.A.K. Diane2

1,2 MIREA – Russian Technological University (Moscow, Russia)

1 a.voronkov.rtu@yandex.ru, 2 sekoudiane1990@gmail.com

Increasing the degree of autonomy and generalizing abilities of information and control systems of manipulative robots, on the one hand, contributes to the expansion of the scope of application of robotic complexes that include manipulators in their composition, and on the other hand, determines the relevance and practical possibility of developing such systems based on modern hardware and software. The task of grasping objects with a parallel gripper is well studied in the scientific literature, while grasping objects with multi-fingered grippers is a more complex task, requiring the calculation of contact points and the calculation of contact interaction forces for reliable object capture.

The article considers the problem of grasping a priori unknown objects using a multi-fingered gripper device. A new approach using the neural network mechanism of attention is proposed to solve the task of grasping a priori unknown objects with the fingertips of a three-fingered gripper device. The approach uses an empirical method to find contact points with their subsequent analytical evaluation based on classical grasping theory. Structurally, the algorithm consists of the following elements: construction and filtering of the target object's point cloud, selection of contact points using a neural net-work with a transformer architecture, assessment of kinematic reachability, force closure and the presence of collisions, grasp execution using the normal force in the feedback loop. Due to its ability to identify features through the use of neural network mechanisms of self-attention and cross-attention, the neural network is able to analyze the local and global context in the source data. 117 and 10 various objects for training and test parts of the dataset, respectively, were used to train the neural network.

According to the results of model experiments in the PyBullet virtual modeling environment, the approach showed more than 90% successful grasps using the beam search and multiple greedy algorithm strategies with a time spent on computing a neural network of 38.4 and 78.4 ms, respectively. The developed software and algorithmic complex allows searching for grasping configurations for the SDH Schunk 2.0 gripper device based on the selected contact points. The results of this study can be applied in areas such as emergency rescue operations, automated storage facilities and service robotics.

Voronkov A.D., Diane S.A.K. Unknown objects grasp planning for multi-finger gripper based on attention mechanism. Neurocomputers. 2024. V. 26. № 5. Р. 80-95. DOI: https://doi.org/10.18127/j19998554-202405-08 (In Russian)

1. Barsuk I.V., Popova E.S. Automated complex of parcels loading in containers. Electromagnetic waves and electronic systems. 2012. V. 17. № 2. P. 67−73. (in Russian)
2. Savrasov G.B., Batanov A.F., Bashlay A.P., Gusarov S.G. Experience in developing of medical robots-manipulators in BMSTU. Biomedicine Radioengineering. 2014. № 10. P. 14–20. (In Russian)
3. Voronkov A.D., Diane S.A.K. Grasping of Unknown Objects with an Autonomous Manipulator: State of the Art, Problems and Prospects. Мechatronics, Automation, Control. 2023. V. 24. № 10. P. 533–541. DOI 10.17587/mau.24.533-541.
4. Xie Zh., Liang X., Roberto C. Learning-based robotic grasping: A review. Frontiers in Robotics and AI. 2023. V. 10. DOI 10.3389/frobt. 2023.1038658.
5. Kleeberger K., Bormann R., Kraus W., Huber M. A Survey on Learning-Based Robotic Grasping. Current Robotics Reports. 2020. V. 1. P. 239–249. DOI 10.1007/s43154-020-00021-6.
6. Platt R. Grasp Learning: Models, Methods, and Performance. Annual Review of Control, Robotics, and Autonomous Systems. 2023. V. 6. № 1. P. 363–389. DOI 10.1146/annurev-control-062122-025215.
7. Breyer M., Chung J., Ott L., Siegwart R., Nieto J. Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Clutter. 4th Conference on Robot Learning (CoRL 2020). 2020. P. 1602–1611.
8. Lundell J., Corona E., Nguyen T., Verdoja F., Weinzaepfel P., Rogez G., Moreno-Noguer F., Kyrki V. Multi-FinGAN: Generative Coarse-To-Fine Sampling of Multi-Finger Grasps. IEEE International Conference on Robotics and Automation (ICRA). 2021. P. 4495–4501.
9. Mayer V., Feng Q., Deng J., Shi Y., Chen Z., Knoll A. FFHNet: Generating Multi-Fingered Robotic Grasps for Unknown Objects in Real-time. International Conference on Robotics and Automation (ICRA). 2022. P. 762–769.
10. Prokudin S., Lassner C., Romero J. Efficient Learning on Point Clouds with Basis Point Sets. Proceedings of the IEEE International Conference on Computer Vision. 2019. P. 4332–4341.
11. Shao L., Ferreira F., Jorda M., Nambiar V., Luo J., Solowjow E., Ojea J.A., Khatib O., Bohg J. UniGrasp: Learning a Unified Model to Grasp with Multifingered Robotic Hands. IEEE Robotics and Automation Letters. 2020. V. 5. № 2. P. 2286–2293. DOI 10.1109/LRA. 2020.2969946.
12. Sundermeyer M., Mousavian A, Triebel R., Fox D. Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes. IEEE International Conference on Robotics and Automation (ICRA). 2021. P. 13438–13444.
13. Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser L., Polosukhin I. Attention is all you need. Advances in Neural Information Processing Systems. 2017. P. 6000–6010.
14. Wang Sh., Zhou Zh., Kan Zh. When Transformer Meets Robotic Grasping: Exploits Context for Efficient Grasp Detection. IEEE Robotics and Automation Letters. 2022. V. 7. № 3. P. 8170–8177. DOI 10.1109/lra.2022.3187261.
15. Zhao Z., Yu H., Wu H., Zhang X. 6-DoF Robotic Grasping with Transformer. [Electronic resource] – Access mode: https://arxiv.org/pdf/ 2301.12476, date of reference 21.05.2024.
16. Lu Y., Fan Y., Deng B., Liu F., Li Y., Wang S. VL-Grasp: a 6-Dof Interactive Grasp Policy for Language-Oriented Objects in Cluttered Indoor Scenes. International Conference on Intelligent Robots and Systems (IROS). 2023. P. 976–983.
17. Lu Y., Deng B., Wang Z., Zhi P., Li Y., Wang S. Hybrid physical metric for 6-dof grasp pose detection. International Conference on Robotics and Automation (ICRA). 2022. P. 8238–8244.
18. Hang K., Stork J., Pokorny F., Kragic D. Combinatorial Optimization for Hierarchical Contact-level Grasping. IEEE International Conference on Robotics and Automation (ICRA). 2014. P. 381–388.
19. Xavier B., Laurent T. The transformer network for the traveling salesman problem. [Electronic resource] – Access mode: https://arxiv.org/pdf/2103.03012, date of reference 21.05.2024.
20. Eldar Y., Lindenbaum M., Porat M., Zeevi Y. The Farthest Point Strategy for Progressive Image Sampling. IEEE Transactions on Image Processing. 1997. V. 6. № 9. P. 1305–1315. DOI 10.1109/83.623193.
21. Calli B., Singh A., Walsman A., Srinivasa S., Abbeel P., Dollar A. The YCB Object and Model Set: Towards Common Benchmarks for Manipulation Research. Proceedings of the IEEE International Conference on Advanced Robotics (ICAR). 2015. P. 510–517.
22. Downs L., Francis A., Joenig N., Kinman B., Kichman R., Reymann K., McHugh T., Vanhoucke V. Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items. International Conference on Robotics and Automation (ICRA). 2022. P. 2553–2560.
23. Leon B., Morales A., Joaquin S. From Robot to Human Grasping Simulation. Springer International Publishing Switzerland. 2014. 261 p.
24. Siciliano B., Khatib O. Springer handbook of robotics. Springer. 2016.
25. Patankar A., Fakhari A., Chakraborty N. Hand-Object Contact Force Synthesis for Manipulating Objects by Exploiting Environment. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2020. P. 9182–9189.
26. Mammadov T.A. Control of the brush-hand system according to the velocity vector. Youth scientific and technical Bulletin. 2015. № 10. P. 7. (in Russian)
27. Haviland J., Corke P. Manipulator Differential Kinematics: Part I: Kinematics, Velocity, and Applications. IEEE Robotics & Automation Magazine. 2023. P. 2–12. DOI 10.1109/MRA.2023.3270228.
28. Wampler C.W. Manipulator inverse kinematic solutions based on vector formulations and damped least-squares methods. IEEE Transactions on Systems, Man and Cybernetics. 1986. V. 16. № 1. P. 93–101. DOI 10.1109/TSMC.1986.289285.
29. Vinyals O., Fortunato M., Jaitly N. Pointer Networks. Advances in Neural Information Processing Systems. 2015. P. 2692–2700.
30. Voronkov A.D. Segmentation of a point cloud with unknown objects using the VCCS method and a dynamic graph convolutional neural network. IT-Standard. 2023. № 4(37). P. 25–35. (in Russian)