Idea Vault: production RAG and agentic AI platform with human-approved edits and multi-provider LLM failover

Idea Vault was a self-initiated project built around one question: can an AI assistant over a user’s own content be trustworthy enough for real use? Typical RAG chatbots struggle with weak retrieval on follow-up questions, slow responses when every query triggers retrieval, outages when a single LLM provider fails, and a safety gap when agents can modify data directly. The goal was a deployed system that stays accurate across multi-turn conversations, responds quickly, keeps working when a provider goes down, and never changes user data without explicit consent.

Designed, built, and deployed solo across 5 iterations (FastAPI, Next.js, MongoDB, Redis, Docker) under a self-authored engineering charter for correctness, reliability, and observability. Retrieval: multi-turn RAG using Hugging Face embeddings over MongoDB Atlas Vector Search (HNSW), with intent classification, history-aware query rewriting, and rolling-summary conversation memory in Redis. Safe agentic edits: the LLM proposes changes through tool calling. They appear as diffs and execute only on explicit user approval, so the model can never modify data directly. Speed and reliability: tiered LLM routing by query intent, with cross-provider failover (Cerebras, Groq, OpenRouter). Simple queries take a fast path (p50 ~220 ms), and retrieval runs only on search queries (p50 ~340 ms), verified in production Langfuse traces. Safety and observability: three-layer topic guardrails block off-topic and code-generation requests. Langfuse and Sentry provide tracing and error monitoring.

Measured results from the production deployment: Speed: simple queries take a fast path with a p50 latency of about 220 ms. Queries that need retrieval run through vector search with a p50 of about 340 ms. Both figures were verified in production Langfuse traces. Reliability: cross-provider LLM failover (Cerebras, then Groq, then OpenRouter) keeps the assistant working when a single provider is unavailable. Risk reduction: the model cannot modify user data directly. Every proposed edit is shown as a diff and executes only after explicit approval. Three-layer topic guardrails block off-topic and code-generation requests. Quality and AI performance: history-aware query rewriting and rolling-summary memory in Redis keep multi-turn conversations grounded, and tiered routing by query intent avoids unnecessary retrieval and model cost. Observability: Langfuse tracing and Sentry error monitoring make behaviour inspectable and regressions easy to spot.