MSc Thesis Proposal: An Approach of Lip Synchronization with Facial Expression Rendering for ECA By: Mohammed Maaz Khan

Tuesday, June 13, 2023 - 01:00 to 02:00

The School of Computer Science is pleased to present…

An Approach of Lip Synchronization with Facial Expression Rendering for ECA

MSc Thesis Proposal by:

Name: Mohammed Maaz Khan


Date: Tuesday, June 13th,2023

Time:  01:00 PM – 02:00 PM

Location: Essex Hall, Room 122



E-Tutoring systems have emerged as effective tools for remote learning and personalized education. However, to foster engaging and interactive experiences, there is a need to enhance the communication and expressiveness of virtual tutors within these systems. This research focuses on integrating facial expression animation and lip-syncing capabilities into an e-tutoring system, aiming to improve the realism and effectiveness of virtual tutor interactions. 

The study presents a novel approach to animating the facial expressions of the virtual tutor's avatar using computer graphics, 3D animation, and computer vision techniques. Working with 3D files and using them to interpolate each face vertex are employed to track and map the facial movements onto the avatar accurately. This enables the virtual tutor to display various facial expressions, including happiness, surprise, and concern, to convey emotions and engage with the learner effectively. Additionally, the system incorporates lip-syncing techniques to synchronize the avatar's lip movements with the spoken content. By analyzing the audio input, the system identifies the phonemes and maps them to the corresponding facial movements, ensuring realistic lip-syncing and enhancing the perception of natural speech. 


Keywords: E-tutoring system, facial expression animation, lip syncing, virtual tutor, 3D animation.


Thesis Committee:

Internal Reader: Dr. Olena Syrotkina 

External Reader: Dr. Leo Oriet          

Advisor: Dr. Xiaobu Yuan