Friday, July 23, 2021 - 11:00 to 13:00
SCHOOL OF COMPUTER SCIENCE
The School of Computer Science is pleased to present…
MSc Thesis Defense by: Manil Snehalbhai Patel
Date: Friday, July 23, 2021
Time: 11:00 AM to 1:00 PM
Meeting URL: https://zoom.us/j/98691626655?from=addon
Passcode: If interested in attending this event, contact the Graduate Secretary at email@example.com
Aspect Based Opinion Mining (ABOM) systems take user's reviews such as "The pizza was delicious, but the staff members were horrible to us." as input from product review site or posts from social media. The system aims to extract the aspect terms (e.g., pizza, staffmember) and categories (e.g., food, service) and their polarities (e.g., Positive, Negative, and Neutral), to help the customers and companies to identify productweaknesses. By solving these product weaknesses, companies can enhance customer satisfaction, increase sales and boost revenues. Neural networks are widely used as classification algorithms for performing ABOM tasks for both the training (learning) phase from historical reviews to form class labels and the testing phase to predict the label for unknown data (new reviews). Neural network algorithms consist of artificial neurons(mathematical functions) that combine input weights (models)and input data (e.g., Review)to predict networkoutputs (e.g., Pizza) through repeated adjustments of prediction errors and back propagated input weights.
Previous approaches, such as BERT-PT (Bidirectional Encoder Representation from Transformers - Post Training) and BAT (BERT Adversarial Training), perform ABOM on User's Reviews by building separate models to complete each ABOM subtasks, e.g., aspect term extraction (e.g., pizza, staff member), and aspect sentiment classification. Their methods can be summarized in steps of obtaining user's reviews, BERT tokenize function (split the sentence into words), BERT encoder layer (convert words into a vector),classification layers (Fully connected neural network to convert vectors into class probability) to perform the Sequence Labeling (extracts the word and word phrases such as pizza and staff members) and multi-Class classification (classify the user's reviews into positive, negative, or neutral). BERT-LSTM/Attention approach also uses different pooling strategies on all the intermediate layers of the BERT model to achieve better results; and does not extract the subtasks of aspect categories nor their related opinion polarities. While BERT-PT uses post-training, BAT uses adversarial training. Their limitation is that they use separate models to perform each ABOM subtasks and require more training time, while not able to consider aspect category and category related opinion polarity tasks.
This thesis proposes the BERT-MTL, which uses the Multi-Task Learning approach, solving two or more tasks simultaneously by taking advantage of the similarities between the tasks and enhancing the model's accuracy by reducing training time. We propose BERT- MTL, which takes a four-step process of obtaining the sentences or user reviews as input, splitting each review into word tokens with the BERT-tokenizer module. The tokens generated are then fed into the BERT encoder layer, which produces each token-related vector. In the end, vectors are then given input to the different classification layers to perform ABOM tasks. To evaluate our model's performance, we have used the SemEval-14 restaurant dataset. Our proposed model (BERT-MTL) outperforms previous models on several ABOM tasks in terms of Macro F1 and Accuracy with the help of Multi-Task Learning and various PoolingStrategies.
Keywords: Aspect Based Opinion Mining,Multi-Task Learning, BERT, Pooling Strategies
MSc Thesis Committee:
Internal Reader: Dr. Saeed Samet
External Reader: Dr. Yahong Zhang
Advisor: Dr. Christie Ezeife
Chair: Dr. Abedalrhman Alkhateeb
MSc Thesis Defense Announcement
5113 Lambton Tower 401 Sunset Ave. Windsor ON, N9B 3P4 (519) 253-3000 Ext. 3716 firstname.lastname@example.org (working remotely)