Summary: The Senior Data Engineer role in Manchester involves supporting a media client's Product Group by enhancing their MAP platform and Redshift Data Warehouse. The position requires implementing critical upgrades to alleviate platform strain, with a focus on data pipeline optimization. The role is a 6-month contract, requiring one day on-site per week, and is classified as inside IR35.

Key Responsibilities:

• Assist a team of Data Engineers in upgrading the MAP platform and Redshift Data Warehouse.
• Implement critical upgrades to reduce strain on existing systems.
• Build and optimize data pipelines using Python.
• Utilize AWS services such as S3, EC2, and EKS.
• Orchestrate workflows using Airflow.
• Employ containerization technologies including Docker and Kubernetes.
• Engage in feature engineering, data pre-processing, and data pipeline development.
• Apply knowledge of relational and non-relational databases, data modeling, and design principles.
• Implement Test-Driven Development (TDD) and adhere to testing best practices.

Key Skills:

• Strong proficiency in Python for data pipeline development.
• Expertise with AWS services (S3, EC2, EKS).
• Experience with Airflow for workflow orchestration.
• Proficiency in Docker and Kubernetes for containerization.
• Experience in feature engineering and data pre-processing.
• Solid understanding of relational and non-relational databases.
• Familiarity with Test-Driven Development (TDD).

Salary (Rate): 500

City: Manchester

Country: United Kingdom

Working Arrangements: hybrid

IR35 Status: inside IR35

Seniority Level: Senior

Industry: IT