Implementation of Retrieval Augmented Generation (RAG) in the Design of Digestive Health Chatbot

Abstract

The development of artificial intelligence technology, especially in the development of chatbots, has brought significant progress, especially in the health sector. However, the main challenge in using large language models (LLM) is the potential for bias and lack of accuracy in providing information, especially on critical topics such as digestive health. This study aims to implement Retrieval-Augmented Generation (RAG) in designing a digestive health chatbot to improve the accuracy and relevance of the information delivered. The RAG method integrates a generative model with a document-based retrieval system to provide more reliable and evidence-based answers. The research process involves collecting digestive health datasets through data scraping from Alodokter, as well as data processing through the preprocessing stage, embedding using the Indonesian language model (firqaaa/indo-sentence-bert-base), and data processing using a vector database with the HNSW index. The Llama 3.1:8b model is used to generate generative responses. The results of the study show that the application of RAG can reduce model bias and improve the quality of chatbot responses. Evaluation using metrics such as Mean Reciprocal Rank (MRR) 93%, Faithfullness 62%, Answer Relevancy 57%, and Semantic Similarity 81% showed good performance in providing accurate and relevant answers according to context. With this approach, chatbots are able to provide more accurate and contextual information according to user needs, and can reduce the risk of hallucinations in the information provided. This research contributes to the development of more reliable health chatbot technology, especially in the digestive health domain, and opens up opportunities for further application in other health fields