About this role
The team is seeking a Senior Data Engineer with strong expertise in Apache Spark, Scala, Python, and GitHub Copilot. The ideal candidate will be responsible for developing scalable data processing solutions and building efficient data pipelines. You will also optimize Spark workloads and leverage AI-assisted development tools to enhance engineering productivity.
Key Responsibilities:
- Design and implement scalable data processing solutions using Apache Spark.
- Develop and maintain robust data pipelines to ensure data integrity and availability.
- Optimize Spark workloads for performance and efficiency.
- Utilize GitHub Copilot to assist in coding and improve development processes.
- Collaborate with cross-functional teams to gather requirements and deliver solutions.
- Monitor and troubleshoot data processing systems to ensure optimal performance.
Required Skills & Qualifications:
- Strong experience with Apache Spark, Scala, and Python.
- Familiarity with GitHub Copilot and its application in software development.
- Proven ability to design and implement data pipelines.
- Experience in optimizing data processing workloads.
- Excellent problem-solving skills and attention to detail.
- Strong communication and collaboration skills.
Experience: 4-8 years in data engineering or related fields.
What we offer:
- Opportunity to work on cutting-edge technology projects.
- Collaborative and innovative work environment.
- Professional development and growth opportunities.
Applications are read by our talent team, usually within two working days.
If you look like a fit we will call you, and you will hear from us either way.