Senior AI Product Analyst - Voice AI Agents

About the role

MEDvidi is an AI-powered mental healthcare platform that combines licensed providers with proprietary AI tools to treat conditions such as ADHD, anxiety and depression. Its AI Receptionist team builds and improves Voice AI Agents that talk directly with patients. This role evaluates, monitors and improves Voice AI systems at scale, working at the intersection of Product, Engineering, Operations and AI. It is fully remote.

Key responsibilities

• Analyze production behavior of Voice AI Agents and conversational AI workflows
• Investigate hallucinations, inconsistencies, silent failures and quality degradation in AI conversations
• Work with call transcripts, logs, generated outputs and operational datasets to find patterns and insights
• Design evaluation frameworks for AI quality, reliability and business impact
• Build monitoring systems and dashboards for AI performance visibility
• Measure the impact of AI systems on efficiency, business outcomes and patient experience
• Detect quality shifts, anomalies and emerging failure modes in production
• Partner with Product, Engineering, QA and Operations teams on AI initiatives

Required profile

• 5+ years in Product Analytics, Data Analytics, Operational Analytics or similar analytics roles in complex production systems
• Fluent Russian; English B2 or higher
• Ability to work CET-overlapping hours
• Strong communication skills and ability to turn analysis into product decisions
• Comfortable with ambiguous problems and fast-changing environments

Required skills

• Strong SQL
• Python for analytical investigations and large-scale text datasets
• Experience with ML pipelines or AI-powered products
• Understanding of LLM limitations, hallucinations, prompt sensitivity and AI failure modes
• Building monitoring systems, analytical frameworks or dashboards
• Nice to have: conversational AI, LLM output evaluation, AI quality metrics and prompt iteration

What we offer

• Competitive compensation and clear growth opportunities
• Fully remote, long-term B2B collaboration
• Health insurance after the probation period, plus sports and wellness compensation
• Career growth, including personalized English lessons via Preply
• 19 working days of vacation per year, 4 extra wellness days and paid sick leave for the first 5 working days
• Gifts for important life events