Comparative Evaluation of LLM-Based Approaches to Chatbot Creation: Implementing a Death Doula Chatbot
Borek, Cecylia (2024)
2024
Master's Programme in Computing Sciences
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2024-02-22
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202402142288
https://urn.fi/URN:NBN:fi:tuni-202402142288
Tiivistelmä
The recent emergence of Large Language Models (LLMs) significantly improved the performance of AI conversational assistants. Creating task-specific chatbots became more accessible with techniques such as Retrieval Augmented Generation (RAG) and fine-tuning. Despite this, the area of creating LLM-based chatbots remains relatively unexplored.
This thesis aimed to compare various approaches to creating LLM-based assistants. Additionally, the objective was to compare the impact of RAG and fine-tuning on the style of responses generated by the model. Finally, this thesis investigated if LLMs are suitable for creating a personal and trustworthy death doula chatbot.
The study encompassed the implementation of a death doula chatbot with four different approaches and an evaluation from the developer and end-user perspective.
The thesis results revealed that open-source approaches, such as fine-tuning Llama2 or using a vector database to build custom RAG, offer the highest level of security, flexibility and control. On the other hand, using well-known platforms, such as OpenAI, is easier and requires no specialised knowledge. Fine-tuning is a better approach for creating chatbots with personality and combining a fine-tuned model with RAG yields the best results. Finally, it was proved that using a fine-tuned LLM, augmented with RAG, holds the potential for creating a personal death doula chatbot with LLMs.
This thesis aimed to compare various approaches to creating LLM-based assistants. Additionally, the objective was to compare the impact of RAG and fine-tuning on the style of responses generated by the model. Finally, this thesis investigated if LLMs are suitable for creating a personal and trustworthy death doula chatbot.
The study encompassed the implementation of a death doula chatbot with four different approaches and an evaluation from the developer and end-user perspective.
The thesis results revealed that open-source approaches, such as fine-tuning Llama2 or using a vector database to build custom RAG, offer the highest level of security, flexibility and control. On the other hand, using well-known platforms, such as OpenAI, is easier and requires no specialised knowledge. Fine-tuning is a better approach for creating chatbots with personality and combining a fine-tuned model with RAG yields the best results. Finally, it was proved that using a fine-tuned LLM, augmented with RAG, holds the potential for creating a personal death doula chatbot with LLMs.