Detection System Indonesian Sign Language (BISINDO) in Video with YOLOv7

Abstract

Sign language is a method of communication that does not use sound but uses physical movements such as hands, body, and lips. One of the sign languages that many deaf people use to communicate is Indonesian Sign Language (BISINDO). This study aims to implement an object detection and image classification model for BISINDO alphabet gestures using the You Only Look Once (YOLO) version 7 algorithm. This research uses video image data consisting of 26 alphabet letters. In this study, three experiments were conducted with different parameter values. Evaluation was carried out using the metrics of mean Average Precision (mAP), precision, recall, and F1-Score. Based on the experiments conducted, the best accuracy was obtained in experiment 1 with parameter values of epoch = 100, batch size = 64, learning rate = 0.001, weight decay = 0.0001, and momentum = 0.9, resulting in mAP@IoU 0.5 value of 0.995, recall 1.00, precision 1.00, F1-Score 1.00. However, it was found that in the application of the model to real-time scenarios, the detection results were not as good as the results obtained during the training process.