How do you store document tokens and vectors in RAG on AWS? Do you combine them in one database or store separately (e.g., S3) while efficiently linking metadata?
In my experience with AWS-based RAG systems, separating document storage from vector storage provides distinct advantages. Document tokens are often best stored as text files or in a NoSQL database, particularly when determining context and ranking relevance, whereas vectors tend to perform better when housed in specialized vector databases optimized for high-dimensional queries. I have found that using S3 for raw document storage while indexing corresponding metadata in a relational or NoSQL database and linking vectors in a dedicated database component ensures performance and scalability. This modular approach simplifies maintenance and improves retrieval efficiency.