MSc Thesis Proposal of Manil Snehalbhai Patel:"Neural Network-based Multi-Task Learning for Product Opinion Mining"

 

Date:  Thursday, April 15th, 2021                

Time:  2:00 pm – 4:00 pm 

Meeting URL: https://zoom.us/j/95245193042?from=addon

Passcode: If interested in attending the event, contact the graduate secretary at csgradinfo@uwindsor.ca

 

 

 

Aspect Based Opinion Mining (ABOM) systems take user's reviews as input such as "The pizza was delicious, but the staff members were horrible to us." from product review site or posts from social media. The system aims to extract the aspect terms (e.g., pizza, staff member) and categories (e.g., food, service) and their polarities (e.g., Positive, Negative, and Neutral), which can help the customers and companies to identify product weaknesses. By solving these product weaknesses, companies can enhance customer satisfaction, increase sales and boost revenues. Neural networks are widely used as classification algorithms for performing ABOM tasks for both the training (learning) phase from historical reviews to form class labels and the testing phase to predict the label for unknown data (new reviews). Neural network algorithms consist of artificial neurons (mathematical functions) that combine input weights (models) and input data (eg. Review) to predict network outputs (eg. Pizza) through repeated adjustments of prediction errors and back propagated input weights. 

 

The previous approaches, such as BERT-PT (Bidirectional Encoder Representation from Transformers - Post Training) and BAT (BERT Adversarial Training), have been proposed to perform ABOM on User's Reviews. These approaches build separate models to complete each ABOM subtasks, such as aspect term extraction (e.g., pizza, staff member) and aspect sentiment classification. The procedure for approaches can be defined in steps of obtaining user's reviews, BERT tokenize function (split the sentence into words), BERT encoder layer (convert words into a vector), classification layers (Fully connected neural network to convert vectors into class probability) to perform the Sequence Labeling (extracts the word and word phrases such as pizza and staff members) and Multi-Class classification (classify the user's reviews into positive, negative, or neutral). However, they differ in their training techniques, where BERT-PT uses post-training and BAT uses adversarial training. The limitation of the previous approaches is that they use separate models to perform each ABOM subtasks. Hence, they require more training time and do not consider the tasks related to aspect category and category related opinion polarity.   

 

This thesis proposes the BERT-MTL, which uses the Multi-Task Learning approach, which differentiates the previous methods by solving two or more tasks simultaneously by taking advantage of the similarities between the tasks and enhancing the model's accuracy by reducing the training time. We propose BERT-MTL, which takes a four-step process, firstly obtaining the sentences or user reviews as input. Secondly, each review is split into tokens (separating each sentence into a list of words) by the BERT-tokenizer module. The tokens generated are then fed into the BERT encoder layer, which produces each token-related vector. In the end, vectors are then given input to the different classification layers to performs ABOM tasks (Aspect terms and categories extraction). To evaluate our model's performance, we have used the SemEval-14 restaurant dataset. Our proposed model (BERT-MTL) outperforms previous models on several ABOM tasks. 

 

Keywords: Aspect Based Opinion Mining, Product Aspect and Opinion Extraction, Multi-Task Learning, Neural Network  

 

Internal Reader: Dr. Saeed Samet 

External Reader:  Dr. Yahong Zhang 

Advisor: Dr. Christie Ezeife