Big Data Engineer

Big Data Engineer

For a leading organization in the Banking sector, we are seeking a skilled Big Data Engineer to join dynamic team. This role is pivotal in developing and maintaining robust data pipelines, ensuring seamless data flow and integration across various platforms. The ideal candidate will possess extensive experience in Scala and PySpark, with a strong background in working with Hadoop and Cloudera platforms. Our company values innovation and efficiency, and we are committed to providing opportunities for professional growth and development. We are looking for someone who can contribute to our agile environment and collaborate effectively with cross-functional teams

Open to New Zealand citizens returning or relocating back to NZ

Your Skills

• Proficiency in building data pipelines with Scala and PySpark, with over five years of experience
• At least one year of experience in managing workflows with Airflow
• Familiarity with Hadoop and Cloudera platforms, particularly in handling Parquet files
• Strong communication skills to effectively collaborate with team members
• Experience in an agile environment, including task creation and management

What You Will Do

• Develop and maintain data pipelines using Scala and PySpark to ensure efficient data processing
• Collaborate with teams to implement and manage workflows using Airflow
• Work with Hadoop and Cloudera platforms, focusing on Parquet file management
• Participate in agile processes, including creating story points and tasks
• Utilize Git and Jira for version control and project management