Senior AI Product Analyst - Automation Team

About the role

MEDvidi's Automation team builds and runs AI and conventional product features that automate clinical and operational workflows, such as AI-drafted medical documentation, automated patient issue resolution, intake verification and provider tools. It is a full-stack product team that ships UI, backend workflows, integrations and AI features together, and measures them the same way. The team is hiring a Product Analyst to own analytics end-to-end as its automated surface area grows.

Key responsibilities

• Own the full product analytics stack: discovery and sizing, experiment design and analysis, funnel and adoption analysis, and post-release impact assessment, across AI-powered and conventional features.
• Analyze production behavior of AI and automation workflows across clinical and operational journeys.
• Investigate hallucinations, silent failures and quality degradation in AI-generated output using logs, transcripts, generated artifacts and operational data.
• Design evaluations for AI quality, both automated and human review, and feed results back into prompt and product changes.
• Build monitoring and dashboards for AI and automation health and their operational metrics.
• Measure the business and operational impact of automation initiatives and surface the risks they introduce.
• Partner with Product, Engineering, Clinical, MedOps and Legal to support fast decisions under ambiguity.

Required profile

• 5+ years of experience in Product Analytics, Operational Analytics or Data Analytics, or analytics roles in complex production systems.
• Understanding of AI, ML and LLM systems; hands-on experience evaluating LLM output (prompt iteration, eval frameworks, human review pipelines) is a strong plus.
• Fluent Russian; English at B2 level or higher.
• Comfortable working overlapping CET hours in a B2B format.
• Nice to have: experience evaluating LLM outputs or AI systems, and previous Data Science or Machine Learning experience.

Required skills

• Strong product and operational analytics background, owning metrics end-to-end.
• SQL and Python coding against messy production data.
• Effective written and verbal communication, turning analysis into decision-ready reports.

What we offer

• Fully remote, long-term B2B collaboration.
• Health insurance after the probation period, plus sports and wellness compensation.
• Career growth, including personalized English lessons via Preply.
• 19 working days of vacation per year, 4 additional wellness days and paid sick leave for the first 5 working days.
• Thoughtful gifts for important life events.