SONAR|HES-SO

SONAR|HES-SO

SONAR|HES-SO regroupe les travaux de bachelor et master diffusables de plusieurs écoles de la HES-SO. Consultez cette page pour le détails.

En cas de question, merci de contacter les bibliothécaires de la HES-SO : bibliotheques(at)hes-so.ch

Back to SONAR
Back
IsmaelJaggi_BT_LLM.pdf
Bachelor thesis

Towards Large Language Models for robotic nutritional coach

  • Sierre : Haute Ecole de Gestion Valais, 2023

Bachelor of Science HES-SO (BSc) in Business Information Technology: School of Management

Large Language Model
Speech-to-Text
Text-to-Speech
Robot
Nutrition
QTrobot
English The objective of this bachelor thesis is to integrate a Large Language Model (LLM) into a QTrobot to create a nutritional coach. In this endeavour, the unique capabilities of the robot, such as facial expressions, gestures, and speech, are combined with the generated responses of the Large Language Model to ensure efficient and user-friendly coaching.
The initial phase involves providing an overview of LLMs and robots already employing LLMs. Subsequently, various open-source LLMs are introduced and compared to identify the most suitable one for the project. The selection process led to the adoption of Llama 2 from MetaAI due to multiple factors.
Given the computational limitations of the robot, preventing the local installation of LLM, alternative solutions were explored. DeepInfra with the Llama 2 70B API emerged as the optimal choice owing to its ease of use and cost-effectiveness.
The implementation on the robot commenced with the creation of a user profile and a login. Subsequently, the two main functions, nutritional coaching, and meal suggestions, were implemented using the DeepInfra API. To enhance user interaction, gestures, facial expressions, and emotions were integrated using the Google Cloud Sentiment API and the functionalities of the robot. Since the program is intended to function in a terminal without a robot, a Platform Abstraction Layer was incorporated. Additionally, in the terminal version, Google Cloud Text-to-Speech (TTS) and Speech-to-Text (STT) APIs were utilized to improve the user experience.
Throughout the implementation phase, test scenarios were devised and conducted three times: twice on the robot and once for the terminal version. To gain an external perspective on the outcome, a user test was conducted at the conclusion of the project.
Language
  • English
Classification
Computer science and technology
Notes
  • Haute Ecole de Gestion Valais
  • Informatique de gestion - Wirtschaftsinformatik
  • hesso:hegvs
Persistent URL
https://sonar.ch/hesso/documents/327438
Statistics
Document views: 222 File downloads:
  • IsmaelJaggi_BT_LLM.pdf: 409