Senior AI Engineer with Databricks

About the role

We are seeking a Senior AI Engineer with deep expertise in the Databricks platform to design, deploy and maintain scalable machine learning pipelines. You will work remotely from anywhere in Georgia, collaborating with cross‑functional teams to deliver production‑ready ML solutions.

Key responsibilities

• Design, implement and maintain end‑to‑end ML pipelines on Databricks.
• Build data ingestion, preprocessing, feature engineering, training and inference workflows using PySpark and Spark ML.
• Manage model versioning, experiment tracking and reproducibility with MLflow.
• Package and deploy models for batch and real‑time inference, monitoring performance and drift.
• Develop scalable ETL/ELT pipelines leveraging Databricks Delta Lake, partitioning, Z‑ordering and caching.
• Integrate with Azure Data Lake, S3, APIs and databases.
• Implement CI/CD pipelines for ML workflows using Azure DevOps, GitHub Actions and Databricks Repos/Jobs API.
• Configure clusters, autoscaling and cost optimization, applying IaC with Terraform, ARM and Bicep.
• Set up logging, alerting and observability to ensure high availability.

Required profile

• 3+ years of experience in machine learning engineering or related roles.
• Proven expertise with the Databricks platform, Apache Spark and PySpark.
• Strong Python skills, including pandas and scikit‑learn.
• Hands‑on experience with MLflow, Docker, REST APIs and Git.
• Solid background in Azure services (Azure Databricks, ADLS, ACR, AML).
• English proficiency at B2 level or higher.

Required skills

• Databricks
• Apache Spark
• PySpark
• Python
• pandas
• scikit‑learn
• MLflow
• CI/CD
• Docker
• REST APIs
• Git
• Azure
• Azure Databricks
• Azure Data Lake
• Azure Container Registry
• Azure Machine Learning
• Azure DevOps
• GitHub Actions
• Terraform
• ARM
• Bicep
• Prometheus
• Grafana
• AWS
• S3
• EMR
• SageMaker
• Spark Structured Streaming
• Databricks Feature Store
• Kubernetes
• XGBoost
• LightGBM
• CatBoost
• Delta Lake

What we offer

• Fully remote work within Georgia, with optional office spaces in Tbilisi, Batumi and Kutaisi.
• Opportunities to work on innovative AI projects for global clients.
• Professional development programs, certifications (GCP, Azure, AWS) and leadership training.
• Corporate and social events, plus the chance to work abroad up to two months per year.
• Relocation possibilities across EPAM’s network in 55+ countries.