Abstract

Effective communication helps in conveying messages with clarity and purpose. Sign language is used by the hard-of-hearing community to communicate using hand signs along with lip movement, facial expression, and body pose. Sign language is not universal and differs from country to country. British Sign Language used in the UK is recognised as an official language. There is a disproportionate number of registered interpreters to the BSL users which calls for the importance of having an electronic recognition system bridging the communication gap between hard-of-hearing and non-signers.

A real-time vision-based sign language recognition system is developed using the techniques of computer vision and deep learning architecture to recognise fingerspelling and six BSL words. The proposed system used OpenCv for capturing the frames, MediaPipe framework to extract the features of the hands and built the LSTM models to predict the signs. The data is collected and trained for both left-hand and right-hand signers. The overall testing accuracy of 26 fingerspelling is 94.23% and six words is 99.07%. Real-time testing accuracy of 86% is obtained for fingerspelling. A graphical user interface is developed for the signers to interact with the developed system for real-time signing.

‘Hand Chat’- British Sign Language Recognition System