Standard top-k retrieval often returns near-duplicate chunks. Interviewers ask how you ensure the LLM sees diverse perspectives, especially for multi-hop or comparative questions.
Askgenai In Answered question
- Maximal Marginal Relevance (MMR): Trade off relevance vs. diversity. Select chunks that are relevant but dissimilar to already-selected ones.
- Clustering-Based Retrieval: Cluster chunks by topic, pick representative from each cluster.
- Submodular Selection: Frame diversity as coverage optimization — greedily pick chunks that cover the most unique information.
- Re-ranking with Diversity: Use multi-objective rerankers that score both relevance and novelty.
- Query Decomposition: Break complex queries into sub-queries, retrieve for each independently, then merge.
Askgenai In Answered question