Closed-loop adversarial ML for payment fraud detection

Fraud detectors are evaluated against the attacks their authors already thought of. A detector can post strong aggregate metrics while carrying a generalization gap that an adversary will find first. A static test set cannot surface that gap, because by definition it sits in the region nobody sampled. What was needed was an evaluation loop that searches for the detector's weaknesses rather than confirming its strengths.

Built a closed Identify / Generate / Defend loop for the Mastercard Innovation Challenge 2026. A NetworkX knowledge graph of payment-fraud vectors drives an attack simulator; generated attacks are scored by an XGBoost detector trained on tabular plus graph features; missed detections feed back as harder variants, so the detector's blind spots determine the next attack batch rather than a fixed test set. The loop runs to convergence and reports the gap it found, the retraining applied, and whether performance elsewhere regressed. Deployed on Cloud Run with a live demo and a FastAPI service behind it.

- Across three independent runs the loop surfaced a real generalization gap every time. - Retraining closed each gap with no regression elsewhere. - 5 of 5 hand-crafted evasions were caught by the retrained detector. - Which gap surfaces varies by run, so the writeup reports all three runs rather than selecting the best one. - Graph features combined with tabular features let the detector reason over relationships between entities, not just transaction attributes. - Live demo deployed on Cloud Run.