On-Premise RAG Assistant & Real-Time IoT Dashboard for Environmental Analytics

Organizations dealing with large-scale environmental and device IoT data often face two main challenges: handling massive streams of real-time time-series data efficiently, and enabling non-technical users to query complex technical documentation securely. Traditional cloud-based LLM APIs introduce significant operational costs, high latency, and data privacy risks when processing sensitive infrastructure data. The system required a highly optimized, fully on-premise architecture that could serve intelligent queries alongside real-time data contexts without compromising speed or exposing data to external networks

Designed and built an end-to-end architecture using Next.js and Golang Gin, coupled with PostgreSQL and TimescaleDB for time-series data pipelines. For the AI capabilities, I deployed a secure local LLM processing unit using Qwen-7B optimized with llama.cpp. I engineered an advanced RAG pipeline utilizing Qdrant vector database for intelligent document indexing and semantic search, enhanced by Cross-Encoder reranking for high answer accuracy. To achieve production-grade efficiency, I optimized the FastAPI RAG backend, successfully driving down query latency from 13.5 seconds to just 1.2 seconds, with local LLM inference hitting 0.04s

The deployment successfully demonstrated that enterprise-grade AI assistants can be run securely and efficiently on-premise without heavy cloud operational costs. By optimizing the FastAPI RAG workflow and fine-tuning llama.cpp for Qwen-7B, the system's end-to-end latency was drastically reduced from an unusable 13.5 seconds to a production-ready 1.2 seconds, with raw LLM inference clocking in at 0.04s. Combined with Qdrant vector indexing and Cross-Encoder reranking, the platform delivered highly accurate technical answers while maintaining 100% data privacy and handling live IoT data streams in real time."