Senior/Lead Data Engineer – Azure, Spark, Python

About the role

Exadel is seeking a Senior/Lead Data Engineer to join a high‑performing team that builds AI‑driven tax‑service tools for a global professional services firm. You will work on designing, developing, and deploying scalable data pipelines and analytics solutions using modern cloud and big‑data technologies.

Key responsibilities

• Define and enforce coding standards and best practices across the project.
• Conduct thorough code reviews and ensure high code quality.
• Design, develop, and maintain robust Spark applications on Databricks.
• Write clean, efficient Python code for complex data manipulation.
• Optimize Spark performance and manage Delta tables, clusters, and workflows.
• Collaborate with cross‑functional teams and communicate regularly with the client.
• Create and maintain documentation for code, processes, and workflows.

Required profile

• Extensive hands‑on experience in software development, especially with Python.
• Proven expertise in PySpark, Spark SQL, and migrating SQL stored procedures to Spark environments.
• Strong background in Azure cloud services and data storage formats such as Parquet and Delta.
• Ability to work independently and lead technical discussions.

Required skills

• Python
• PySpark / Spark
• Databricks
• Azure Cloud
• .NET 8 and ASP.NET Core
• MongoDB
• Azure SQL
• Angular
• Kendo UI
• GitHub Enterprise
• Copilot
• LangGraph
• LangChain
• RAG Pipelines
• Multi‑modal LLMs
• Polars
• Pandas
• Parquet
• Delta Tables