Sinhala Sign Language Translation using Transfer using Transfer Learning

Abstract

The automated interpretation of sign language is a challenging task to proceed, the sign recognitions required high level computer vision and the high level motion capturing and motion processing system to generate more accurate results in image perception. In previous years there was a number of research done for the communication between deaf and mute people with the society in the international context. ASL, ISL, and other sign language translation to proper text or voice from different technologies. In this study focus on the Sri Lankan Sign Language translation and making effective communication between the deaf and mute people and the society. After studying the previous technologies which focus on machine learning image classification and identification, get the successful point as well as the failures done in this research.In this thesis, the author used machine learning and transfer learning to make a system which is capable of translating Sinhala Signs into text. The model consists of the implementation of a pre-trained MobileNetV2 model. The created modelrelies on the transfer learning during the training of model and data. The research has specifically recognized 5 sinhala signs. The model was hosted on a platform and the users can use it with the web application interface which provides all the input of images as well as provide real time sign translations. The TensorFlow JS based hosted model creates json as output for the web application user end. Users can translate the Sinhala Signs real time which is more useful in the effective communication between the people who are using the Sinhala sign language for their communication. The system is capable enough for work in any kind of system which is performing heterogeneous behaviour in the context.