MSc Thesis Defense " An Approach of Lip Synchronization with Facial Expression Rendering for ECA " By: Mohammed Maaz Khan

Tuesday, September 5, 2023 - 09:00 to 10:00

The School of Computer Science at the University of Windsor is pleased to present …

An Approach of Lip Synchronization with Facial Expression Rendering for ECA

MSc Thesis Defense by: Mohammed Maaz Khan

Date:  September 5th, 2023

Time:  9:00AM - 10:00AM

Location: Essex Hall, Room #105

Abstract: E-Tutoring systems have emerged as effective tools for remote learning and personalized education. However, to foster engaging and interactive experiences, there is a need to enhance the communication and expressiveness of virtual tutors within these systems. This research focuses on integrating facial expression animation and lip-syncing capabilities into an e-tutoring system, aiming to improve the realism and effectiveness of virtual tutor interactions. 

The study presents a novel approach to animating the facial expressions of the virtual tutor's avatar using computer graphics, 3D animation, and computer vision techniques. Working with 3D files and using them to interpolate each face vertex are employed to track and map the facial movements onto the avatar accurately. This enables the virtual tutor to display various facial expressions, including happiness, surprise, and concern, to convey emotions and engage with the learner effectively. Additionally, the system incorporates lip-syncing techniques to synchronize the avatar's lip movements with the spoken content. By analyzing the audio input, the system identifies the phonemes and maps them to the corresponding facial movements, ensuring realistic lip-syncing and enhancing the perception of natural speech. 

Keywords: E-tutoring system, facial expression animation, lip syncing, virtual tutor, 3D animation.

MSc Thesis Committee:
Internal Reader:
Dr. Olena Syrotkina      

External Reader: Dr. Leo Oriet
Advisor: Dr. Xiaobu Yuan
Chair: Dr. Robin Gras