Data Engineer

Summary: The role of Data Engineer involves designing, developing, and optimizing data pipelines using Azure Databricks and Scala. The position requires collaboration with team members to meet project requirements and support customer engagements, ensuring high-performance solutions for large datasets. The engineer will also communicate project status and address any issues that arise during the process.

Key Responsibilities:

• Develop data engineering solutions using Azure Databricks and Scala.
• Build and optimize complex data pipelines handling large datasets (millions of records).
• Collaborate with team members and participate in customer discussions as needed.
• Communicate project status and escalate issues when necessary.

Key Skills:

• Strong experience in Azure Data Factory, Databricks, and data pipeline development.
• Proficient in Scala programming.
• Experience with performance optimization for large datasets.
• Good understanding of Azure services and problem-solving skills.
• Strong collaboration and communication skills.

Salary (Rate): undetermined

City: London

Country: UK

Working Arrangements: undetermined

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT

We are looking for a Data Engineer to design, develop, and optimize data pipelines using Azure Databricks and Scala. You will collaborate with the team to meet project requirements, support customer engagements, and ensure high-performance solutions for large datasets.

Key Responsibilities:

• Develop data engineering solutions using Azure Databricks and Scala.

• Build and optimize complex data pipelines handling large datasets (millions of records).

• Collaborate with team members and participate in customer discussions as needed.

• Communicate project status and escalate issues when necessary.

Skills & Experience:

• Strong experience in Azure Data Factory, Databricks, and data pipeline development.

• Proficient in Scala programming.

• Experience with performance optimization for large datasets.

• Good understanding of Azure services and problem-solving skills.

• Strong collaboration and communication skills.

Nice to Have:

• Agile team experience and understanding of Agile practices.

• Databricks certification.

• Knowledge of CI/CD pipelines.