Senior/Lead Data Engineer – Azure Data & AI

About the role

Exadel is looking for a Senior/Lead Data Engineer to join a high‑performing team that builds AI‑driven tax‑service tools for a global professional services firm. You will work on Azure‑based micro‑services, Spark applications and modern data pipelines, turning complex analytical challenges into scalable solutions.

Key responsibilities

• Define and enforce coding standards and best practices across the project.
• Conduct thorough code reviews to maintain high code quality.
• Design, develop, and maintain robust, scalable Spark and PySpark applications on Databricks.
• Write clean, efficient Python code for complex data manipulation using Pandas or Polars.
• Optimize performance and scalability of Spark jobs, including Delta Lake management.
• Collaborate closely with cross‑functional teams and communicate regularly with the client.
• Identify and resolve technical issues, ensuring reliability of applications.
• Create and maintain comprehensive documentation for code, processes, and workflows.

Required profile

• Extensive hands‑on experience in software development.
• Proven expertise developing SQL stored procedures and migrating them to Spark SQL or PySpark.
• Deep experience with PySpark on Databricks, including Delta tables, cluster management and workflow automation.
• Track record of optimizing Spark performance in production environments.
• Strong proficiency in Python for data manipulation (Pandas, Polars).
• Solid understanding of columnar storage formats, especially Parquet and Delta Lake.

Required skills

• Python
• PySpark
• Apache Spark
• Databricks
• Delta Lake / Delta tables
• SQL / Spark SQL
• Pandas
• Polars
• Parquet
• Azure Cloud
• .NET 8 & ASP.NET Core
• MongoDB
• Azure SQL
• Angular
• Kendo UI
• GitHub Enterprise & Copilot
• LangGraph
• LangChain
• RAG pipelines
• Multi‑modal LLMs