Netflow: Automated Prepaid Utility Metering with Real-Time Anomaly Detection

Building managers tracked electricity and water usage through manual monthly meter readings and manually disconnected supply when a resident's prepaid credit ran out. This was labor-intensive, error-prone, and gave no visibility into meter tampering, reverse-flow, or sensor faults until a manual inspection caught them.

Honestly: this isn't a machine learning system, and that was a deliberate engineering call, not a limitation I'm glossing over. The team weighed three detection paradigms, deterministic rules, statistical rolling-window thresholds, and machine learning, against two hard requirements. F-07 required an instant alert on a live threat like forced reconnection or an unsafe voltage spike, not a pattern learned after the fact. NF-08 required logic that could be tested exhaustively with a light compute footprint on high-frequency telemetry from thousands of devices. A rules engine let us hit both: 40 deterministic unit tests cover all 8 anomaly conditions with perfect precision and recall on that test set, and the same logic runs in well under the 90-second alert budget because there's no model inference in the path.

There was also no labeled training data. This is a new deployment at a single building, so there was no historical corpus of confirmed tampering or leakage incidents to train or validate a model against, and shipping an unvalidated model in a system that can trigger an automatic supply cutoff felt like the wrong trade for a first release.

The roadmap is explicit about where ML does belong here: augmenting the rule-based engine with a model, an LSTM is the candidate in my thesis, to catch nonlinear per-unit tampering patterns that fixed thresholds can't, while keeping the deterministic rules in place as an auditable safety net underneath it. That is a staged decision to introduce AI once there's real labeled production data to justify it, not a rejection of AI as a direction.

Six software decisions shaped this system, each weighed against a named requirement rather than picked by default:

Anomaly detection: deterministic rules per anomaly type, not a statistical rolling-window model or ML, because F-07 (instant alert on a live threat) and NF-08 (exhaustively testable logic) ruled out anything probabilistic for a first release.

Session management: stateless JWT instead of server-side sessions, so any server instance can validate a session independently once the service scales horizontally (NF-09, F-14).

Identity verification: delegated to Google OAuth instead of owning password storage, moving credential storage and recovery risk outside the system (F-11, NF-01).

Async processing: an in-memory queue (BullMQ on Redis) instead of a full message broker or a distributed log, sized for the lightest operational footprint that still fits one building's load (F-23, NF-07). The report's own scaling roadmap calls out Kafka as the upgrade path once the system spans many buildings, not before.

Telemetry storage: a partitioned relational database (MySQL, monthly partitions) instead of a dedicated time-series database, so the team handing the system over operates one engine instead of two (NF-09, NF-10).

OTA update layout: dual-bank A/B firmware instead of single-bank, so a failed update on a fleet of 2,500+ devices can roll back automatically instead of bricking a unit's meter (F-23).

The hardware-level decision underneath all of this used Analytical Hierarchy Process scoring across 4 controller placement alternatives against 4 weighted criteria (economic, sustainability, environmental, manufacturability). The winning alternative, one controller per unit handling both electricity and water, was not the top scorer on any single criterion, but it produced the simplest software data model: a strict one-to-one mapping between unit and device, so detection, billing, and cutoff logic never needed a merge or fan-out step. That mapping is the reason the software side of the six decisions above stayed as simple as it did.

Scalability and observability mechanisms worth noting: monthly telemetry partitioning, a bounded connection pool (DBPOOLSIZE=60), BullMQ back-pressure, a streaming async generator for whole-fleet operations so memory use stays flat regardless of unit count, pruning of in-memory anomaly state past 5,000 entries, a dedicated EMQX MQTT broker separate from the application server, OpenTelemetry distributed tracing, 5-component health checks, and an immutable audit log.

Several concrete bugs shaped the final design rather than staying theoretical:

The live consumption chart went blank in testing because the nginx MQTT proxy pointed at the wrong broker target and the fleet simulator was sending under the wrong device ID. Fixed by repointing the proxy to broker.emqx.io and correcting the device ID, but it's the reason device identity validation (F-15) became an explicit, tested requirement rather than an assumption.

The anomaly endpoint returned the wrong data types for backflow and zero-flow cases, caught by the type system once the 40-case unit test suite was in place, and fixed with corrected type casts.

A resident-creation endpoint accepted unvalidated input; class-validator decorators were added to close it, which became the template for validating every other DTO in the system.

The payment flow failed silently because nothing was listening for the confirmation event; wiring up @nestjs/event-emitter fixed the immediate bug and prompted an audit of every other place a step could fail without a visible signal, which is part of why the 5-component health check and immutable audit log exist.

At the infrastructure level, the MQTT broker's Last-Will-Testament setting was blocking device reconnection, and one device was pointed at an insecure production URL. Fixing both surfaced the weakness that firmware still runs with MQTTTLSINSECURE=1, certificate verification off, which the thesis names directly as unfinished work rather than something discovered after the fact and hidden.

Beyond individual bugs, the evaluation itself is honest about its limits. Precision and recall of 1.00/1.00 hold on a curated 40-case deterministic test set, not on independently labeled production incidents, because none existed yet for a brand-new single-site deployment. Detection thresholds (voltage range, MCB power limit, alert de-duplication window) are stated as engineering defaults, not values tuned against real incident data, and the report documents that as a threat to validity instead of presenting the numbers as more final than they are.

This is a capstone project moving into handover, not a system with months of live end-user operation behind it yet, and I want that stated plainly rather than implied away. Resident and building-manager adoption in the everyday sense, training sessions, feedback loops, usage metrics, hasn't happened yet.

What we did build for adoption was the handover itself, treated as a deliverable rather than an afterthought. The system goes to two audiences who are not me: the building's non-technical management staff, and whatever developer maintains this next. For staff, an in-app FAQ page explains features and expected behavior in plain language, so day-to-day questions don't require calling the original team. For the next developer, backend and frontend README files cover setup and structure, a dbnotes.md file records why the schema looks the way it does, and CLAUDE.md and AGENTS.md files give any future developer, human or an AI coding assistant, the project context and hard constraints up front instead of leaving them to reverse-engineer it. A full analysis-through-testing document trail (internally labeled B100 through B500) preserves the reasoning behind decisions, not just the decisions themselves.

The change-management thinking behind the design was to remove steps for end users, not add them. A resident's actual task barely changes: top up credit and check consumption. What changes is that this now happens digitally and at any hour, instead of during counter hours or after chasing down a photographed meter reading. Building management goes from finding out about a leak or tampering a month later to seeing an alert in under 90 seconds, which is a bigger process change for them than for the residents. Measuring how that management-side change actually lands once the system runs for months, not weeks, is explicitly next, not something I'm claiming already happened.

Augment the deterministic engine with a machine learning model, an LSTM is the candidate in the thesis, to catch nonlinear per-unit tampering patterns that fixed thresholds miss, while keeping the current rule-based logic in place underneath it as an auditable safety net rather than replacing it outright.

Move the async path from Redis and BullMQ to Apache Kafka once the system needs to span many buildings instead of one, for higher partitioned throughput than an in-memory queue was ever designed to provide.

Apply mutual TLS on the MQTT path so both broker and device require certificates, closing a specific known gap: current firmware runs with MQTTTLSINSECURE=1, so certificate verification is off in production today, and that is the single item I would prioritize first before any of the scale-driven work below.

Follow the staged scalability roadmap already written into the report rather than over-building now: read replicas when the deployment grows from one building to several, table sharding by building plus message broker clustering at city scale, and extracting any given module into its own service only once real load proves it is a bottleneck, not preemptively.

Replace the curated 40-case test set with precision and recall measured against actual labeled production incidents once the system has enough operating history to have some, which is the honest next step for the detection numbers reported here.

Pick the simplest method that actually meets the stated latency and testability requirement, not the one that sounds most impressive on a resume. Deterministic rules let this system hit a hard sub-90-second alert budget with logic that's fully reproducible and covered by 40 passing unit tests. An unvalidated first ML model, with no labeled production data to train or check it against, almost certainly couldn't have matched that on day one, and this system can trigger an automatic supply cutoff, which is not where I wanted to learn that the hard way.

Second: make every requirement traceable back to a real quote from the people who'll actually use the system, not an assumption about what they probably need. All 27 functional and 12 non-functional requirements in this project link directly to a specific line from field interviews with the building manager and the owners' association. That discipline paid for itself at evaluation time. Every success criterion had one obvious way to verify it, because it was written to be checked from the start rather than interpreted after the fact.

Third, and the one I'd push hardest on: say what you didn't measure. Our evaluation states plainly that precision and recall of 1.00 apply to our own curated test set, not independently labeled field incidents, and that the team who built the system is also the team who evaluated it. That's a more useful signal to anyone deciding whether to trust this work than a clean number with the caveats left out, and it's the difference between a claim and an evidenced one, which is presumably the entire point of a platform built around evidence rather than adjectives.