Data Engineer - Python & Azure
Posted 1 week ago by TieTalent on Linkedin
Negotiable
Undetermined
Hybrid
Leeds, England, United Kingdom
Summary: The Data Engineer role focuses on managing and optimizing LCCC's data ecosystems, collaborating with various stakeholders to define and implement an effective data platform. The position involves building ETL/ELT pipelines, ensuring data integrity, and enhancing data quality for internal forecasting models related to GB power markets. The role requires hands-on experience with Azure and Python, along with a strong understanding of data governance and architecture. The position is based in Leeds or Birmingham, with a hybrid working arrangement.
Key Responsibilities:
- Build and maintain ETL/ELT pipelines to make data accurate and easy to use
- Ingest and transform data sets from various sources including APIs, web scraping, and third-party services
- Analyze data to identify patterns and anomalies in preparation for ETL processes
- Ensure efficient data synchronization and flow between platforms and systems
- Enhance data quality and reliability
- Establish a data culture across the organization
- Drive better data governance through principles and processes
- Set service level indicators and monitor execution of data workflows
- Identify data quality issues through profiling and stakeholder engagement
Key Skills:
- At least 1 year of data engineering experience
- Hands-on experience within the Azure Data ecosystem (Azure Databricks, Data Factory, Data Lake, Synapse)
- Strong competence in Python, ideally with PySpark experience
- Strong competence in SQL
- Experience building and maintaining data pipelines
- Experience managing DevOps Pipelines
- Strong experience in process optimization, performance tuning, and data modeling
Salary (Rate): 61000
City: Leeds
Country: United Kingdom
Working Arrangements: hybrid
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT