Dynamic Robotic Grasping: A Combination of Real-Time Trajectory Planning and ML-Based Novel Object Pose Detection

Huy Hoang Nguyen1       Minh Nhat Vu2,4       Anh Nguyen3       Zoltán Istenes1      
1Eötvös Loránd University   2ACIN - TU Wien   3University of Liverpool   4Austrian Institute of Technology  
Video

Abstract

This thesis explores integrating real-time grasping with 6-DoF pose estimation and online trajectory optimization to improve performance in challenging conditions like poor lighting and novel objects, crucial for industrial manufacturing. The D435i camera was chosen for its superior performance. Using Optitrack for ground truth, the study found the Foundation pose method more accurate and reliable than the Aruco method under static and dynamic conditions. Integrating online trajectory optimization improved grasping and placement, addressing camera calibration and frame transformation issues. Future work includes real-time object detection and language-guided grasping commands to enhance system utility. This research demonstrates the potential of advanced pose estimation and trajectory planning technologies to enhance industrial automation with more precise and adaptive robotic interactions.

Video

Experiment in Realsense D400 Series

Results

results1

Results from our method show that the Foundation pose trajectory is notably more accurate and consistent than the Aruco system's trajectory in dynamic conditions. This makes our method more suitable for applications requiring precise and reliable motion tracking. The Foundation trajectory displays a high level of consistency, with slight variances observed potentially within acceptable ranges for many applications. Compared to the previously analyzed Aruco trajectory, the Foundation pose shows significant improvement in maintaining trajectory alignment with the ground truth. The Aruco system, while effective in certain aspects, shows potential weaknesses in dealing with dynamic tracking, as indicated by the noisier and more fluctuating trajectory. This might limit its use in applications requiring high precision and stability.



results2

Furthermore, our approach shows robust performance in tracking translation, particularly along the Z-axis, where errors are minimal, suggesting notable altitude stability. Nevertheless, greater variability in translation errors on the X and Y axes and significant orientation errors across all axes, underscores the system's challenges in managing complex rotations and rapid directional changes. These aspects are critical for applications that demand precise depth measurements, such as augmented reality or robotic navigation.



Acknowledgements

I would like to extend my profound gratitude to my supervisor, Dr. techn. Minh Nhat Vu, for his unwavering support and invaluable lessons throughout my research journey. His remarkable advice has significantly enriched my knowledge and experience, contributing immensely to the timely completion of my thesis with outstanding results. Additionally, I am grateful to Assoc. Prof. Zoltán Istenes and Assoc. Prof. Anh Nguyen for their guidance and insights, which have been pivotal in shaping the direction of my work. I also wish to acknowledge my colleague at TU Wien, whose assistance and collaboration were instrumental during the project. Most importantly, I am deeply thankful for the unwavering support from my family, whose encouragement has been my cornerstone. I am eager to further explore the field of robotics, with the hope of uncovering new insights and developments.

We borrow the page template from  Nerfies project page. Special thanks to them!
This website is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.