About this role
The team is seeking a highly skilled Senior Data Engineer with strong experience in Azure, Databricks, and large-scale data processing to design, build, and optimize enterprise-grade data solutions. This hands-on role requires deep technical expertise in data architecture, distributed processing, and cloud-native data platforms, along with exposure to AI/ML data use cases and modern Lakehouse practices. The ideal candidate will play a key role in developing scalable, secure, and reliable data pipelines and platforms that enable analytics, reporting, and AI-driven business capabilities.
Key Responsibilities:
- Design, develop, and maintain scalable data architecture and data engineering solutions on Azure and Databricks.
- Build robust batch and streaming data pipelines using Apache Spark, PySpark, Delta Lake, SQL, and related technologies.
- Develop and optimize complex data processing frameworks for structured, semi-structured, and high-volume datasets.
- Implement enterprise data lakehouse solutions aligned to business, analytics, and AI/ML requirements.
- Ensure data pipelines are reliable, performant, reusable, and built with strong engineering standards.
- Collaborate with business teams, data architects, analysts, data scientists, and cloud engineers to translate requirements into technical solutions.
- Support AI and ML initiatives by creating curated, governed, and feature-ready datasets.
- Work on performance tuning, troubleshooting, and optimization of Databricks workloads and Spark jobs.
- Establish and maintain data quality, lineage, security, governance, and compliance controls.
- Drive best practices in CI/CD, code quality, testing, release management, and operational support for data solutions.
- Contribute to platform improvements, architecture reviews, and technology evaluation activities.
Requirements:
- Bachelor's or master's degree in computer science, information technology, engineering, or a related field.
- 6-8 years of experience in data engineering, data platform engineering, or related roles.
- Strong hands-on experience with Azure cloud services and Azure Databricks.
- Deep expertise in Apache Spark / PySpark, Delta Lake, SQL, and distributed data processing.
- Strong understanding of data architecture, data modeling, and modern lakehouse patterns.
- Experience building and supporting complex ETL/ELT pipelines in enterprise environments.
- Working exposure to AI and ML data pipelines, feature preparation, or data support for AI-based solutions.
- Experience with batch and streaming data processing.
- Solid understanding of data governance, security, access control, and data quality practices.
- Experience with orchestration tools such as Azure Data Factory, Databricks Workflows, Airflow, or similar.
- Strong debugging, analytical thinking, and problem-solving skills.
What we offer:
The team provides a dynamic work environment that encourages innovation and professional growth. You will have the opportunity to work on cutting-edge technologies and contribute to impactful projects in the data engineering space.