MSc Thesis Defense: Enhancing Composed Image Retrieval with Constraint Verification by Tina Aminian

Enhancing Composed Image Retrieval with Constraint Verification

MSc Thesis Defense by: Tina Aminian

Date: 2026/03/30

Time: 12-2 PM

Location: EH186

 

Composed Image Retrieval (CIR) aims to retrieve images from a vector database that best match a user’s intent, expressed through a reference image together with a textual description specifying desired modifications. This retrieval setting supports applications such as e-commerce search and general image retrieval. Recent CIR models typically fuse the visual reference and textual modification into a shared embedding space, where similarity is computed for retrieval. From the user-centric perspective, the two components of the user’s query serve fundamentally

different roles: the reference image acts as a soft semantic anchor, while the text query functions as a mandatory requirement for the expected changes. Because existing neural models treat both inputs symmetrically, the logical constraints expressed in the modification text may be ignored during inference, leading to results that violate the user’s explicit instructions. To address this issue, we propose a novel approach in fashion retrieval to restore the must status of the text query while preserving the expressive power of natural language. Our framework

introduces a verifiable logical structure based on a constraint meta-model for representing user intent. This meta-model is instantiated through a semiautomatically constructed, data-driven ontology of fashion attributes that serves as the domain schema. The model incorporates relational operators to capture attribute-level changes relative to the reference image. We leverage prompt engineering to translate user modification texts into formal constraints defined within this schema. A plug-in verification layer is implemented to filter out candidates that violate the constraints during post-processing, ensuring the mandatory requirements are satisfied while retaining the semantic richness of the underlying

model. Experimental results on fashion benchmarks show that our approach significantly enhances the recall of existing state-of-the-art methods, demonstrating the logical verification can be an essential companion to neural retrieval in fashion e-commerce.

 

Internal Reader: Dr.Lu

Internal Reader: Dr. Asaduzzaman

Advisor: Dr.Chen

Chair: Dr.Saini