Re-embedding millions of documents is expensive. The interviewer wants to hear about delta updates, model versioning, caching, and dimensionality reduction.
Askgenai In Answered question
- Delta Updates: Only embed new/changed documents. Maintain a changelog and versioned index.
- Model Versioning: Keep old index + old embedding model running alongside new. Gradually migrate instead of full re-embedding.
-
Dimensionality Reduction: Use Matryoshka embeddings (OpenAI
text-embedding-3series) — store full vectors but query with shorter dimensions for speed. - Quantization: Convert float32 embeddings to int8 or binary. 4x storage reduction, minimal accuracy loss.
- Lazy Embedding: Don’t embed low-value documents (duplicates, drafts). Use heuristics to filter before embedding.
Askgenai In Answered question