Real-time voice AI interview coach: full-duplex agent with ~1.5s voice-to-voice latency and barge-in

Voice agents feel unnatural when they reply slowly, cut people off mid-thought, or keep talking when interrupted, and LLM feedback can quote things the speaker never said. The goal was a spoken interview-practice coach that converses at human pace, handles pauses and interruptions gracefully, and gives feedback grounded in what the candidate actually said.

Built and deployed a full-duplex, real-time voice agent (Python, FastAPI, WebSockets). - Pipeline: browser audio streamed into NVIDIA Nemotron streaming ASR (Riva), Qwen3 on Groq with reasoning off for instant speakable replies (model chosen from a preference list at startup), and NVIDIA Magpie TTS. - Latency: sentence-by-sentence synthesis and model prewarming. - Turn-taking: adaptive end-of-turn detection (1.3-2.6s) that tolerates thinking pauses, plus ~300ms barge-in that cancels the agent's reply when the user interrupts. - Conversation logic: 3 questions drawn from a bank of 14, follow-ups when answers are thin, and scored feedback on 5 competencies. - Trust: a grounding check verifies every quote the model cites against the live transcript. - Quality: 66 automated tests.

- Voice-to-voice latency cut roughly in half, from ~3s to ~1.5s. - ~300ms barge-in: the agent stops speaking almost immediately when interrupted. - Adaptive end-of-turn window (1.3-2.6s) avoids cutting candidates off mid-answer. - Grounding check rejected 42 of 70 (60%) model-cited quotes that did not match the transcript across two scripted test sessions, preventing fabricated feedback. - Covered by 66 automated tests; live and publicly accessible.