Integrating Deep Learning for Object Manipulation: A 7-DOF Robotic Arm Perspective on Grasping
DOI:
https://doi.org/10.63094/AITUSRJ.25.4.1.5Keywords:
DOF Robotic arm, Deep learning, Object Detection, Object Manipulation and grasping, YOLOAbstract
The Robotic arm with 7-Degree of Freedom (DOF) is extensively used in numerous industrial applications. However, its precision and control need further improvement for optimum results in various generalized applications. This paper presents a novel approach to improve the manipulation capabilities of a 7-DOF robotic arm by integrating the YOLOv7 object detection model and a Deep Reinforcement Learning (DRL) framework for control. YOLOv7 is employed to provide real-time perception, enabling accurate object recognition, while the DRL algorithm optimizes control by adapting to the dynamic environment of the robotic arm. The DRL algorithm learns through trial and error, adapting to the specific dynamics of the robotic arm and its environment. As a result, improved precision, stability, and adaptability were observed across various tasks. The primary contribution of this work is the optimization and integration of YOLOv7 with a Raspberry Pi, facilitating efficient and real-time object manipulation even on resource-constrained hardware. The proposed algorithm was trained on diverse datasets, enabling the system to generalize effectively across multiple objects and real-world scenarios. Extensive experiments, including repeated trials under varying conditions, demonstrated significant improvements in grasping accuracy and manipulation performance compared to traditional control methods. The system achieved a validated accuracy rate of 94%, supported by statistical analysis and confusion matrix evaluation, confirming its robustness and reliability. These results highlight the potential of intelligent robotic arms to perform complex tasks with high precision and adaptability autonomously.
References
D. Kulkarni, Computer vision and fuzzy-neural systems. Prentice Hall PTR, 2001.
M. Ebner, “A parallel algorithm for color constancy,” Journal of Parallel and Distributed Computing, vol. 64, no. 1, pp. 79–88, 2004.
D. Forsyth and J. Ponce, “Prentice hall professional technical reference,” Computer vision: a modern approach, 2002.
C.-Y. Tsai, C.-C. Wong, C.-J. Yu, C.-C. Liu, and T.-Y. Liu, “A hybrid switched reactive-based visual servo control of 5-dof robot manipulators for pick-and-place tasks,” IEEE Systems Journal, vol. 9, no. 1, pp. 119–130, 2015.
G. Bradski, A. Kaehler, and V. Pisarevsky, “Learning- based computer vision with intel’s open source computer vision library.” Intel Technology Journal, vol. 9, no. 2, 2005.
C. H. Lampert, H. Nickisch, and S. Harmeling, “Learning to detect unseen object classes by between- class attribute transfer,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009, pp. 951–958.
Saxena, J. Driemeyer, and A. Y. Ng, “Robotic grasping of novel objects using vision,” The International Journal of Robotics Research, vol. 27, no. 2, pp. 157–173, 2008.
M. Kazemi, K. K. Gupta, and M. Mehrandezh, “Randomized kinodynamic planning for robust visual servoing,” IEEE Transactions on Robotics, vol. 29, no. 5, pp. 1197–1211, 2013.
D. Song, C. H. Ek, K. Huebner, and D. Kragic, “Task- based robot grasp planning using probabilistic inference,” IEEE transactions on robotics, vol. 31, no. 3, pp. 546–561, 2015.
R. Szab´o and A. S. Gontean, “Full 3d robotic arm control with stereo cameras made in labview.” in FedCSIS Position Papers, 2013, pp. 37–42.
Y. Hasuda, S. Ishibashi, H. Kozuka, H. Okano, and J. Ishikawa, “A robot designed to play the game” rock, paper, scissors”,” in Industrial Electronics, 2007. ISIE 2007. IEEE International Symposium on. IEEE, 2007, pp. 2065–2070.
Shaikh, G. Khaladkar, R. Jage, and T. P. J. Taili, “Robotic arm movements wirelessly synchronized with human arm movements using real time image processing,” in India Educators’ Conference (TIIEC), 2013 Texas Instruments. IEEE, 2013, pp. 277–284.
S. Manzoor, R. U. Islam, A. Khalid, A. Samad, and J. Iqbal, “An open-source multi-dof articulated robotic educational platform for autonomous object manipulation,” Robotics and Computer-Integrated Manufacturing, vol. 30, no. 3, pp. 351–362, 2014.
T. P. Cabre, M. T. Cairol, D. F. Calafell, M. T. Ribes, and J. P. Roca, “Project-based learning example: controlling an educational robotic arm with computer vision,” IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, vol. 8, no. 3, pp. 135–142, 2013.
B. Rooks, “The harmonious robot,” Industrial Robot: An International Journal, vol. 33, no. 2, pp. 125–130, 2006.
Barrett Technology, Inc., “WAM Arm,” 2010. [Online]. Available: http://www.barrett.com/robot/products-arm-specifications.html.
Meka Robotics, “A2 compliant arm,” 2009. [Online]. Available: http://www.mekabot.com/arm.html.
R. Brooks, C. Breazeal, M. Marjanovi´c, B. Scassellati, and M. Williamson, “The Cog project: Building a humanoid robot,” Computation for metaphors, analogy, and agents, pp. 52–87, 1999.
Edsinger-Gonzales and J. Weber, “Domo: A force sensing humanoid robot for manipulation research,” in 2004 4th IEEE/RAS International Conference on Humanoid Robots, 2004, pp. 273–291.
E. Torres-Jara, “Obrero: A platform for sensitive manipulation,” in 2005 5th IEEE-RAS International Conference on Humanoid Robots, 2005, pp. 327–332.
H. Iwata, S. Kobashi, T. Aono, and S. Sugano, “Design of anthropomorphic 4-dof tactile interaction manipulator with passive joints,” Intelligent Robots and Systems, 2005 (IROS 2005), pp. 1785 – 1790, Aug. 2005.
J. Pratt, B. Krupp, and C. Morse, “Series elastic actuators for high fidelity force control,” Industrial Robot: An International Journal, vol. 29, no. 3, pp. 234–241, 2002.
M. Zinn, B. Roth, O. Khatib, and J. Salisbury, “A new actuation approach for human friendly robot design,” The international journal of robotics research, vol. 23, no. 4-5, p. 379, 2004.
D. Shin, I. Sardellitti, and O. Khatib, “A hybrid actuation approach for human-friendly robot design,” in IEEE Int. Conf. on Robotics and Automation (ICRA 2008), Pasadena, USA, 2008, pp. 1741–1746.
K. Wyrobek, E. Berger, H. der Loos, and J. Salisbury, “Towards a personal robotics development platform: Rationale and design of an intrinsically safe personal robot,” in Proc. IEEE Int. Conf. on Robotics and Automation, 2008, pp. 2165–2170.
Willow Garage, “PR2,” 2010. [Online]. Available: http://www.willowgarage.com/pages/pr2/specs.
G. Hirzinger, N. Sporer, A. Albu-Schaffer, M. Hahnle, R. Krenn, A. Pascucci, and M. Schedl, “DLR’s torque-controlled light weight robot III- Are we reaching the technological limits now?” in Proceedings- IEEE International Conference on Robotics and Automation, vol. 2, 2002, pp. 1710–1716.
Schunk, “7-DOF LWA Manipulator,” 2010. [Online]. Available: http://www.schunk-modular-robotics.com/left-navigation/service-robotics/components/manipulators.html.
R. Ambrose, H. Aldridge, R. Askew, R. Burridge, W. Bluethmann, M. Diftler, C. Lovchik, D. Magruder, and F. Rehnmark, “Robonaut: NASA’s space humanoid,” IEEE Intelligent Systems and Their Applications, vol. 15, no. 4, pp. 57–63, 2000.
J. Stuckler, M. Schreiber, and S. Behnke, “Dynamaid, an anthropomorphic robot for research on domestic service applications,” in Proc. of the 4th European Conference on Mobile Robots (ECMR), 2009.
KUKA, “youbot arm,” 2010. [Online]. Available: http://www.kuka-youbot.com.
Zhang Q, Zhou J, Wang H, Chai T. 2016. Output feedback stabilization for a class of multi-variable bilinear stochastic systems with stochastic coupling attenuation. IEEE Transactions on Automatic Control 62(6):2936-2942.
Liu P, Yu H, Cang S. 2014. Modelling and control of an elastically joint-actuated cart-pole underactuated system.
Liu P, Yu H, Cang S. 2016. Modelling and dynamic analysis of underactuated capsule systems with friction-induced hysteresis.
Liu P, Yu H, Cang S. 2018a. Geometric analysis-based trajectory planning and control for underactuated capsule systems with viscoelastic property. Transactions of the Institute of Measurement and Control 40(7):2416-2427.
Liu P, Yu H, Cang S. 2018b. On the dynamics of a vibro-driven capsule system. Archive of Applied Mechanics 88(12):2199-2219.
Liu P, Yu H, Cang S. 2018c. Optimized adaptive tracking control for an underactuated vibro-driven capsule system. Nonlinear Dynamics 94(3):1803-1817.
Liu P, Yu H, Cang S. 2018d. Trajectory synthesis and optimization of an underactuated microrobotic system with dynamic constraints and couplings. International Journal of Control, Automation and Systems 16(5):2373-2383.
Liu P, Yu H, Cang S. 2019. Adaptive neural network tracking control for underactuated systems with matched and mismatched disturbances. Nonlinear Dynamics 98(2):1447-1464.
Liu P, Neumann G, Fu Q, Pearson S, Yu H. 2018b. Energy-efficient design and control of a vibro-driven robot.
Huda MN, Liu P, Saha C, Yu H. 2020. Modelling and motion analysis of a pill-sized hybrid capsule robot. Journal of Intelligent & Robotic Systems 100(3–4):753-764.
Cao P, Gan Y, Duan J, Dai X. 2019. Passivity-based stable human-robot cooperation with variable admittance control.
Sugiarto I, Conradt J. 2017. A model-based approach to robot kinematics and control using discrete factor graphs with belief propagation. Robotics and Autonomous Systems 91(9):234-246
Toan NV, Khoi PB. 2019. Fuzzy-based-admittance controller for safe natural human–robot interaction. Advanced Robotics 33(15–16):815-823.
Kang G, Oh HS, Seo JK, Kim U, Choi HR. 2019. Variable admittance control of robot manipulators based on human intention. IEEE/ASME Transactions on Mechatronics 24(3):1023-1032.
Abraham I, Handa A, Ratliff N, Lowrey K, Murphey TD, Fox D. 2020. Model-based generalization under parameter uncertainty using path integral control. IEEE Robotics and Automation Letters 5(2):2864-2871.
Popović M, Kootstra G, Jørgensen JA, Kragic D, Krüger N. 2011. Grasping unknown objects using an early cognitive vision system for general scene understanding.
Williams G, Wagener N, Goldfain B, Drews P, Rehg JM, Boots B, Theodorou EA. 2017. Information theoretic MPC for model-based reinforcement learning.
Wu M, Taetz B, Saraiva ED, Bleser G, Liu S. 2019b. On-line motion prediction and adaptive control in human-robot handover tasks.
Lee MA, Zhu Y, Zachares P, Tan M, Srinivasan K, Savarese S, Fei-Fei L, Garg A, Bohg J. 2020. Making sense of vision and touch: learning multimodal representations for contact-rich tasks. IEEE Transactions on Robotics 36:582-596.
Wang H, Wang Z, Wang H. 2019. Impedance control strategy and experimental analysis of collaborative robots based on torque feedback.
Esmaeili B, Salim M, Baradarannia M, Farzamnia A. 2019. Data-driven observer-based model-free adaptive discrete-time terminal sliding mode control of rigid robot manipulators.
Perrusquía A, Yu W, Soria A. 2019. Optimal contact force of robots in unknown environments using reinforcement learning and model-free controllers.
Wu W, Li D, Meng W, Zuo J, Liu Q, Ai Q. 2019a. Iterative feedback tuning-based model-free adaptive iterative learning control of pneumatic artificial muscle.
Zhou Y, Zhang Q, Wang H, Zhou P, Chai T. 2017. EKF-based enhanced performance controller design for nonlinear stochastic systems. IEEE Transactions on Automatic Control 63(4):1155-1162.
Zhang Q-C, Hu L, Gow J. 2020. Output feedback stabilization for MIMO semi-linear stochastic systems with transient optimisation. International Journal of Automation and Computing 17(1):83-95.
Joseph, R & Ali, F. (2018). YOLOv3: An Incremental Improvement. 1804.02767. https://docs.ultralytics.com/models/yolov3/
Bochkovskiy, A, Wang, C & Mark Liao, H. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. 2004.10934. https://docs.ultralytics.com/models/yolov4/
Jocher, G. (2020). Ultralytics YOLOv5. https://doi.org/10.5281/zenodo.3908559/
Li, C & ET all. (2023). YOLOv6 v3.0: A Full-Scale Reloading. 2301.05586. https://docs.ultralytics.com/models/yolov6/
Wang, Chien-Yao and Bochkovskiy, Alexey and Liao, Hong-Yuan Mark. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. 2207.02696. https://docs.ultralytics.com/models/yolov7/
Jocher, G, Chaurasia, A & Qiu, J. (2023). Ultralytics YOLOv8. https://github.com/ultralytics/ultralytics
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 AITU SCIENTIFIC RESEARCH JOURNAL
This work is licensed under a Creative Commons Attribution 4.0 International License.
