Automated job lead monitoring and AI scoring system. Reduced manual processing time from 20 mins to 30 secs per lead using n8n, Python, and LLMs, eliminating data loss during API rate limits.
Manual processing of incoming job leads took 15 to 20 minutes per request, causing delayed responses and missed opportunities during traffic spikes. The legacy workflow lacked automated deduplication, intelligent prioritization, and robust error handling. When external LLM APIs imposed rate limits (HTTP 429), the system would drop requests or hang, leading to operational bottlenecks and data loss.
Designed and deployed an end-to-end AI automation pipeline. The system ingests leads via RESTful webhooks, processes them through an n8n orchestration layer, and applies LLM-based scoring (Groq API) to prioritize high-value candidates. To handle API rate limits, I implemented a custom Python Code node with exponential backoff and jitter, ensuring zero data loss during 429 errors. Data is deduplicated in PostgreSQL, with real-time alerts sent to a Telegram bot. The architecture is containerized with Docker.
The deployment transformed a manual, bottlenecked process into a fully automated, resilient pipeline. Lead response and scoring time dropped from 15-20 minutes to approximately 30 seconds. The custom retry logic completely eliminated data loss during external API rate limits. The system now stably handles 100+ concurrent daily executions without database locking or performance degradation.
Reduced lead response and scoring time from 15-20 minutes to ~30 seconds per lead. Achieved stable, concurrent processing of 100+ daily executions without database locking or data loss during API rate limits.