MSc Thesis Defense Announcement of Manil Snehalbhai Patel:"Neural Network-based Multi-Task Learning for Product OpinionMining "

 

Date: Friday, July 23, 2021                           

Time: 11:00 AM to 1:00 PM                                           

Passcode: If interested in attending this event, contact the Graduate Secretary at csgradinfo@uwindsor.ca 

 

Aspect Based Opinion Mining (ABOM) systems take user's reviews such as "The pizza was delicious, but the staff members were horrible to us." as input from product review site or posts from social media. The system aims to extract the aspect terms (e.g., pizza, staffmember) and categories (e.g., food, service) and their polarities (e.g., Positive, Negative, and Neutral), to help the customers and companies to identify productweaknesses. By solving these product weaknesses, companies can enhance customer satisfaction, increase sales and boost revenues. Neural networks are widely used as classification algorithms for performing ABOM tasks for both the training (learning) phase from historical reviews to form class labels and the testing phase to predict the label for unknown data (new reviews). Neural network algorithms consist of artificial neurons(mathematical functions) that combine input weights (models)and input data (e.g., Review)to predict networkoutputs (e.g., Pizza) through repeated adjustments of prediction errors and back propagated input weights. 

 

Previous approaches, such as BERT-PT (Bidirectional Encoder Representation from Transformers - Post Training) and BAT (BERT Adversarial Training), perform ABOM on User's Reviews by building separate models to complete each ABOM subtasks, e.g., aspect term extraction (e.g., pizza, staff member), and aspect sentiment classification. Their methods can be summarized in steps of obtaining user's reviews, BERT tokenize function (split the sentence into words), BERT encoder layer (convert words into a vector),classification layers (Fully connected neural network to convert vectors into class probability) to perform the Sequence Labeling (extracts the word and word phrases such as pizza and staff members) and multi-Class classification (classify the user's reviews into positive, negative, or neutral). BERT-LSTM/Attention approach also uses different pooling strategies on all the intermediate layers of the BERT model to achieve better results; and does not extract the subtasks of aspect categories nor their related opinion polarities. While BERT-PT uses post-training, BAT uses adversarial training. Their limitation is that they use separate models to perform each ABOM subtasks and require more training time, while not able to consider aspect category and category related opinion polarity tasks. 

 

This thesis proposes the BERT-MTL, which uses the Multi-Task Learning approach, solving two or more tasks simultaneously by taking advantage of the similarities between the tasks and enhancing the model's accuracy by reducing training time. We propose BERT- MTL, which takes a four-step process of obtaining the sentences or user reviews as input, splitting each review into word tokens with the BERT-tokenizer module. The tokens generated are then fed into the BERT encoder layer, which produces each token-related vector. In the end, vectors are then given input to the different classification layers to perform ABOM tasks. To evaluate our model's performance, we have used the SemEval-14 restaurant dataset. Our proposed model (BERT-MTL) outperforms previous models on several ABOM tasks in terms of Macro F1 and Accuracy with the help of Multi-Task Learning and various PoolingStrategies. 

 

Keywords: Aspect Based Opinion Mining,Multi-Task Learning, BERT, Pooling Strategies 

Internal Reader: Dr. Saeed Samet 

External Reader: Dr. Yahong Zhang 

Advisor: Dr. Christie Ezeife 

Chair: Dr. Abedalrhman Alkhateeb