Augmented Color Input in CIR Requirements
MSc Thesis Defense by: Arnob Banik
Date: Monday, August 17th, 2026
Time: 2PM
Location: Essex Hall 122
Abstract:
Composed Image Retrieval (CIR) enables users to retrieve target images by combining a reference image with a natural-language modification query; however, linguistic color descriptions may be too broad or ambiguous to represent precise color preferences. This thesis proposes a model-independent color-processing layer that augments the conventional CIR query with an optional visual palette input without modifying or retraining the underlying semantic retrieval model. Textual color expressions and palette selections are converted into axis-aligned RGB regions, and their compatibility is evaluated using three-dimensional Intersection over Union. Compatible inputs produce a validated color constraint, whereas conflicting requirements generate a notification and terminate retrieval. Gallery images are represented using semantic embeddings and dominant RGB metadata and organized through a voxelized RGB space. Reusable Hierarchical Navigable Small World indices are constructed for sufficiently populated color partitions, while smaller partitions may be processed using exact scanning. Eligible candidates are validated against the accepted RGB bounds and ranked using semantic similarity combined with a Gaussian RGB-distance score. The framework was evaluated on the FashionIQ dress, shirt, and top-tee categories by comparing color-first voxel prefiltering with global semantic retrieval followed by post-filtering. The experiments varied voxel resolution, exact-scan threshold, HNSW configuration, candidate-processing strategy, post-filter depth, and gallery size. Performance was evaluated using median and tail query latency, time-to-index, and amortized cost. The results show that moderate prebuilt voxel configurations substantially reduce online and amortized processing costs compared with global post-filtering, whereas fine voxelization introduces sparse-partition and fallback overhead. Our findings demonstrate that explicit color constraints can be integrated into CIR without modifying the semantic model and that moderate color-first voxel routing provides a practical retrieval strategy.
Keywords: Composed Image Retrieval, Multimodal Retrieval, Software Requirement Engineering
Thesis Committee:
Internal Reader: Dr. Muhammad Asaduzzaman
Internal Reader 2: Dr. Ikjot Saini
Advisor: Dr. Jessica Chen
Chair: Dr. Xiaobu Yuan
