SCS Colloquium Presentation Announcement of Dr. Caroline Lemieux:"The Promises and Challenges of ML in Automated Software Testing"

Friday, March 24, 2023 - 11:00 to 12:00


Colloquium Presentation by Dr Caroline Lemieux

Picture of a woman smiling
Date: March 24, 2023
Time: 11:00am – 12:00pm
Location: Erie Hall, Room 3123
Reminders: 1. Two-part attendance mandatory (sign-in sheet, QR Code)
2. Arrive 5-10 minutes prior to event starting - LATECOMERS WILL NOT BE ADMITTED. Note that due to demand, if the room has reached capacity, even if you are "early" admission is not guaranteed.
3. Please be respectful of the presenter by NOT knocking on the door for admittance once the door has been closed whether the presentation has begun or not (If the room is at capacity, overflow is not permitted (ie. sitting on floors) as this is a violation of the Fire Safety code).
4. Be respectful of the decision of the advisor/host of the event if you are not given admittance. The School of Computer Science has numerous events occurring soon.


Automated software testing aims to help developers find bugs by providing them with test inputs or test cases that expose unexpected behaviours in a software system under test. Traditionally, random and genetic algorithms have been used for this purpose. This talk explores two works that use Machine Learning to try and improve software testing, and covers both the upsides and downsides of these techniques. First, this talk will touch on work using reinforcement learning to try and improve the generation of valid inputs (RLCheck). Second, the talk will cover recent work that leverages large language models of code, in particular, OpenAI's Codex, to help genetic algorithm generate more realistic test cases test cases. The talk will give both the positive results and the possible drawbacks of these ML techniques for testing.
Keywords: software engineering, software testing, fuzz testing, search-based software test, reinforcement learning, large language models, Codex


Caroline Lemieux is an Assistant Professor of Computer Science at UBC. Her research aims to build tools that improve the correctness, security and performance of software systems, with a focus on innovations in fuzz testing, program synthesis, and specification mining. Her research on fuzz testing has been awarded an ACM SIGSOFT Distinguished Paper Award, ACM SIGSOFT Distinguished Artifact Award, ACM SIGSOFT Tool Demonstration Award, and Industry-Track Best Paper Award. She received her B.Sc. in Combined Honours Computer Science and Mathematics at UBC, where she won the Governor General’s Silver Medal in Science. She completed her PhD at UC Berkeley, where she was the recipient of a Berkeley Fellowship for Graduate Study and a Google PhD Fellowship in Programming Technologies and Software Engineering. Most recently, she explored the use of LLMs for testing as a Postdoctoral Researcher and Microsoft Research.


Vector Institute in Artifical Intelligence, artificial intelligence approved topic logo


5113 Lambton Tower 401 Sunset Ave. Windsor ON, N9B 3P4 (519) 253-3000 Ext. 3716