The Business Challenge
The business situation the organisation faced before the project began.
High-volume inboxes bury urgent messages and deadlines, and manual triage wastes time. Naive AI triage tools call the Gmail API and an LLM for every message, which is slow, costly and hits API quotas, and they rarely isolate data between multiple users.
Why AI Was the Right Solution
Why AI was the right approach — and what alternatives were considered.
Pure rules engines break on the variety of real email: the same request can be phrased countless ways, and deadlines are written in free text. Off-the-shelf filters sort by sender or keyword but do not understand intent or extract dates. An LLM handles that ambiguity well, but using it for every message is slow and wasteful.
The right answer was a hybrid: deterministic rules for the predictable majority, and AI only where language understanding adds value. That combination gives accuracy where it matters while keeping cost and latency low.
How the Solution Was Delivered
Discovery, design, development, testing and rollout — the journey, not the tooling.
- 1Mapped how triage decisions are actually made: what counts as urgent, which senders matter, and where deadlines hide in message text.
- 2Designed a rules-first pipeline so common, predictable cases never need an LLM call.
- 3Added an LLM fallback only for messages the rules could not classify confidently.
- 4Built three entry points for different users: a web dashboard, a Gmail add-on inside the inbox, and a CLI for power users.
- 5Added a feedback loop so user corrections refine future classifications.
- 6Restricted outbound actions to drafts only, keeping a human in control of every reply.
- 7Wrote 59 regression tests and validated the system against real Gmail inboxes before relying on it.
Key Technical & Architecture Decisions
Architecture, model selection, workflow and trade-offs.
Rules first, LLM second: reserving the model for ambiguous cases cut cost and latency and made results more predictable.
Groq for inference: chosen for fast responses so triage feels interactive.
Local embeddings with FastEmbed for semantic search: keeps search fast and avoids sending inbox content to another external service.
OAuth 2.0 with PKCE and per-user data isolation: required for a genuinely multi-user system handling private email.
Drafts-only replies: a deliberate trade-off that gives up full automation in exchange for trust and safety.
Batched Gmail API access: the single biggest efficiency gain, reducing calls by 92.5%.
Challenges & How They Were Solved
Obstacles hit along the way and how they were overcome.
API quotas and latency: calling the Gmail API per message did not scale. Batching requests brought calls down from about 200 to 15 per inbox.
Multi-user privacy: a shared system handling personal email needed strict separation. OAuth 2.0 with PKCE and per-user data isolation ensure each user only ever touches their own data.
LLM unpredictability: model output can vary. Putting rules first, and keeping replies as drafts, limits the impact of any wrong classification or weak reply.
Regressions while iterating: as rules and prompts evolved, earlier behaviour could break. A suite of 59 regression tests plus validation on real inboxes caught this early.
User Adoption & Change Management
How users responded — training, change management and feedback loops.
Trust was the main adoption question, since people are cautious about AI acting on their email. Two design choices addressed that: the system never sends anything on its own, and it learns from the corrections users make, so it gets closer to their judgement over time. Meeting users where they already work, through a Gmail add-on, alongside a dashboard and a CLI, removed the need to change habits to benefit from it.
Business Outcomes & Impact
The measurable outcomes from the underlying AI Deployment — with the story behind them.
Gmail API calls reduced by 92.5% (200 to 15 per inbox) across 1,800+ messages; 59 regression tests passing. Estimated ~15 hours of manual triage saved (assumes ~30 seconds per email across 1,800+ messages).
The story behind the numbers
Batching reduced Gmail API calls from about 200 to 15 per inbox (92.5%) across more than 1,800 messages. That keeps the system well within API quotas, lowers latency and makes it practical to run for several users at once rather than one.
The bigger shift is in how time is spent: instead of reading every message to decide what matters, a user starts from a prioritised view with deadlines already extracted, and reviews suggested replies instead of writing them from scratch. Because replies are drafts only, that time saving comes without giving up control over what is sent.
Lessons Learned
What surprised the team, what worked well, and what would be done differently.
- Rules first, AI second keeps cost and latency down.
- Batching external API calls can matter more than model choice.
- Drafts-only output builds user trust in AI that touches email.
- Per-user isolation is essential for multi-user AI on private data.
- Regression tests make it safe to keep iterating on rules and prompts.
Future Opportunities
Where this solution could go next.
Support for other mail providers beyond Gmail.
Scheduled daily digests summarising priorities and upcoming deadlines.
Calendar integration so extracted deadlines become reminders automatically.
An agentic mode that can handle low-risk, well-understood replies end to end, with clear user-defined limits.
Team inboxes with shared triage rules and routing.
Professional Reflections
Guidance for another builder tackling a similar problem.
Do not reach for an LLM for everything. Start with the cheapest reliable method, usually rules, and bring in the model only where it clearly adds value. Measure external API usage early, because batching and caching often deliver the biggest wins. For anything that acts on personal data, design for trust from the start: keep a human in the loop, isolate user data, and build a regression suite before you start tuning prompts.