• M A Hannan

Augmented Reality in Welding Using Deep Learning Algorithm

Despite significant advances in the digitalization world, developing defects classification and recognition efficiently at scale presents serious challenges to current approaches. As Industry 4.0 is moving towards digitalization, shipyards should consider AR to support welding training and operations. In this research, a common platform is developed to perform welding training in augmented reality and detect welding surface defects.


This application act as a support system to improve training techniques, reduce man-hours and a more effective quality inspection. TensorFlow is adopted for model training to detect defects before transferring the model into Unity 3D. The main challenge for this application is to construct a method that ensures the welding gun can simulate welding with accuracy from its position and movement. The author developed an application in the Unity environment to understand the challenges of such estimation. The significance of this study is to enhance traditional training and a more precise standard of inspection. Welders need to strengthen their knowledge in welding and obtain proper techniques before entering an actual welding booth and day-to-day welding operation.


Python, TensorFlow and Unity 3D, and a working prototype are used to develop this platform.

This developed application will be helpful for training, especially during the pandemics, to lessen the contacts and reduce the number of trainers. However, the user interface and accuracy level can be improved further. Besides, object tracking requires strong pixel detection. Also, Unity mostly supports Android devices, with the latest mobile devices suitable to use ARCore, thus, limiting the usage of the in-house sensors. These will be addressed in the future phases of this research.


Please feel free to watch this video to learn more:


4 views0 comments