Data Engineer - Azure databricks - python - pyspark - Azure Data Factory - Azure Data Lake
Posted Today by Nexere Consulting Limited
£50,000 Per year
Fixed-Term
Remote
United Kingdom/London/Sheffield
Summary: The role of Junior Data Engineer focuses on developing and evolving a cloud-based data platform using Azure technologies. The position involves designing scalable data pipelines and lakehouse solutions to ensure high-quality data availability for analytics and reporting. The role is a fixed-term contract for 12 months, emphasizing collaboration with stakeholders to meet data requirements. The position is fully remote, allowing for flexibility in work arrangements.
Key Responsibilities:
- Design, build and support scalable data platforms, pipelines and lakehouse solutions using Azure Databricks, Azure Data Factory and Python/PySpark.
- Partner with business stakeholders, analysts and technical teams to understand data requirements and deliver robust data solutions that support reporting, analytics and operational needs.
- Develop and maintain data ingestion, transformation and orchestration processes, integrating data from a variety of internal and external sources.
- Ensure data solutions are performant, reliable and maintainable, applying best practices in data engineering, automation and distributed processing.
- Monitor, troubleshoot and optimise data pipelines, proactively addressing issues relating to data quality, availability and performance.
Key Skills:
- Commercial experience in a Data Engineering role, with strong expertise in Python/PySpark and the Azure data ecosystem.
- Hands-on experience with Azure Databricks, Azure Data Factory, Azure SQL, Azure Data Lake and related cloud technologies.
- Strong understanding of data modelling, ETL/ELT development and data integration principles.
- Knowledge of modern cloud-based data architectures, distributed computing concepts and large-scale data processing.
- Experience using Infrastructure as Code tools such as Terraform or Bicep, alongside CI/CD practices and Azure DevOps.
- Excellent communication and stakeholder management skills, with the ability to engage effectively with both technical and business audiences.
Salary (Rate): £50,000 yearly
City: undetermined
Country: United Kingdom
Working Arrangements: remote
IR35 Status: fixed-term
Seniority Level: undetermined
Industry: IT
Data Engineer - Azure databricks - python - pyspark - Azure Data Factory - Azure Data Lake - CI/CD
- FTC role - 12 Months
- Salary - Up to £50,000 + Benefits
- Location - Remote
As a Junior Data Engineer, you will play a key role in the development and evolution of our cloud-based data platform. Leveraging technologies including Azure Databricks, Azure Data Factory and PySpark, you will design and deliver scalable data pipelines and lakehouse solutions that enable trusted, high-quality data across the organisation. Working within a modern Azure environment, you will help ensure data is efficiently managed, readily available and governed to support analytics, reporting and business decision-making.
Key Responsibilities
- Design, build and support scalable data platforms, pipelines and lakehouse solutions using Azure Databricks, Azure Data Factory and Python/PySpark.
- Partner with business stakeholders, analysts and technical teams to understand data requirements and deliver robust data solutions that support reporting, analytics and operational needs.
- Develop and maintain data ingestion, transformation and orchestration processes, integrating data from a variety of internal and external sources.
- Ensure data solutions are performant, reliable and maintainable, applying best practices in data engineering, automation and distributed processing.
- Monitor, troubleshoot and optimise data pipelines, proactively addressing issues relating to data quality, availability and performance.
Skills & Experience
- Commercial experience in a Data Engineering role, with strong expertise in Python/PySpark and the Azure data ecosystem.
- Hands-on experience with Azure Databricks, Azure Data Factory, Azure SQL, Azure Data Lake and related cloud technologies.
- Strong understanding of data modelling, ETL/ELT development and data integration principles.
- Knowledge of modern cloud-based data architectures, distributed computing concepts and large-scale data processing.
- Experience using Infrastructure as Code tools such as Terraform or Bicep, alongside CI/CD practices and Azure DevOps.
- Excellent communication and stakeholder management skills, with the ability to engage effectively with both technical and business audiences.