Enhancing Hierarchical Retrieval Augmented Generation with Structure Aware Retrieval
MSc Thesis Proposal by: George Kaceli
Date: Friday, November 6th
Time: 10:00 am
Location: Essex Hall 122
Abstract:
Retrieval-Augmented Generation (RAG) grounds large language models in external evidence at inference time, but its effectiveness depends heavily on how the underlying corpus is indexed. Flat vector indexes treat each chunk as an independent unit. This suits localized fact lookup, but it discards the thematic, sequential, and hierarchical organization that long-document questions often depend on. Recursive hierarchical indexing addresses this by clustering document chunks, summarizing each cluster, and repeating the process. The result is a tree whose leaves preserve source text and whose upper layers capture progressively broader themes. Two design decisions in this approach remain largely unexamined. The first is the clustering algorithm, which determines which chunks are summarized together and therefore what every higher layer represents. The second is the retrieval policy: common practice flattens the tree into a single pool at query time, discarding the parent–child structure that construction produces. This thesis treats both decisions as experimental variables. It introduces a modular clustering interface through which GMM, K-Means, agglomerative clustering, DBSCAN, and Leiden community detection build trees under otherwise identical conditions. These trees are compared with flat retrieval on three benchmarks that reward different forms of reasoning. The thesis then introduces structure-aware retrieval policies that use the tree's edges at query time. These policies descend from a retrieved summary to its source passages, add a parent summary when retrieved passages converge on it, expand to related passages in the same cluster, or route each question to the level of the tree that suits it. The experiments indicate that the value of hierarchical indexing is conditional, depending on the question type, the clustering method, and how the tree is used. Clustering and retrieval should therefore be designed and evaluated jointly rather than in isolation.
Thesis Committee:
Internal Reader: Dr. Luis Rueda
External Reader: Dr. Ahmed Hamdi Sakr
Advisor: Dr. Jianguo Lu
