Saiful Bahri, Iffah Zulaikha and Saon, Sharifah and Mahamad, Abd Kadir and Isa, Khalid and Fadlilah, Umi and Ahmadon, Mohd Anuaruddin and Yamaguchi, Shingo (2023) Interpretation of Bahasa Isyarat Malaysia (BIM) Using SSD-MobileNet-V2 FPNLite and COCO mAP. Information, 14 (319). pp. 1-20.
![[img]](http://eprints.uthm.edu.my/style/images/fileicons/text.png) |
Text
J16131_fcea86e7763277e20bdbfab7a92ccd21.pdf Restricted to Registered users only Download (897kB) | Request a copy |
Abstract
This research proposes a study on two-way communication between deaf/mute and normal people using an Android application. Despite advancements in technology, there is still a lack of mobile applications that facilitate two-way communication between deaf/mute and normal people, especially by using Bahasa Isyarat Malaysia (BIM). This project consists of three parts: First, we use BIM letters, which enables the recognition of BIM letters and BIM combined letters to form a word. In this part, a MobileNet pre-trained model is implemented to train the model with a total of 87,000 images for 29 classes, with a 10% test size and a 90% training size. The second part is BIM word hand gestures, which consists of five classes that are trained with the SSD-MobileNet-V2 FPNLite 320 × 320 pre-trained model with a speed of 22 s/frame rate and COCO mAP of 22.2, with a total of 500 images for all five classes and first-time training set to 2000 steps, while the second- and third-time training are set to 2500 steps. The third part is Android application development using Android Studio, which contains the features of the BIM letters and BIM word hand gestures, with the trained models converted into TensorFlow Lite. This feature also includes the conversion of speech to text, whereby this feature allows converting speech to text through the Android application. Thus, BIM letters obtain 99.75% accuracy after training the models, while BIM word hand gestures obtain 61.60% accuracy. The suggested system is validated as a result of these simulations and tests.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | Bahasa Isyarat Malaysia (BIM); SSD-MobileNet-V2 FPNLite; COCO mAP; TensorFlow Lite; Android application |
| Subjects: | T Technology > T Technology (General) |
| Divisions: | Faculty of Electrical and Electronic Engineering > FKEE |
| Depositing User: | Mr. Mohamad Zulkhibri Rahmad |
| Date Deposited: | 25 Sep 2024 07:21 |
| Last Modified: | 25 Sep 2024 07:21 |
| URI: | http://eprints.uthm.edu.my/id/eprint/11622 |
Actions (login required)
 |
View Item |