Fake News Detection Model Using BERT and Bi-LSTM Based on Discriminative Approach

Abstract

This study aimed to develop a text classification model for detecting hoaxes using a deep learning approach and text representation methods. The text data that had undergone preprocessing were then extracted using three approaches: Word2Vec, Doc2Vec, and Bidirectional Encoder Representations from Transformers (BERT). The research dataset consisted of 2,325 genuine news articles (label 0) and 2,287 fake news articles (label 1). In this study, BERT feature vectors with a dimension of 768 were combined with the Bidirectional Long Short-Term Memory (Bi-LSTM) algorithm to capture sequential dependencies in the text, along with the Support Vector Machine (SVM) algorithm as the final classifier. The training process was carried out on Dell Precision 7750 hardware using parameters of embedding dimension 128, 64 hidden units, a dropout rate of 0.3, and a learning rate of 0.001. Training and testing were conducted for 10 epochs with a batch size of  32. The results indicated that the Word2Vec and Bi-LSTM model achieved an accuracy of 87.4% with an F1-Score of 87.0%, while the Doc2Vec and Bi- LSTM model performed slightly lower with an accuracy of 85.6% and an F1- Score of 85.4%. The best performance was obtained by the BERT, Bi-LSTM, and SVM model, which achieved an accuracy of 93.8%, precision of 94.1%, recall of 93.5%, and an F1-Score of 93.7%.