Summary: The Senior Data Engineer role is tailored for an experienced professional specializing in AWS-based data solutions and systems engineering. This position involves designing, building, and maintaining scalable data pipelines and architectures while integrating advanced analytics and generative AI to enhance business insights. The candidate will also mentor junior team members and collaborate with cross-functional teams to deliver effective technical solutions.

Key Responsibilities:

• Develop and maintain scalable, high-performance ETL pipelines using AWS services such as Lambda, Glue, Lake Formation, Athena, and EMR.
• Architect data warehousing and data lake solutions to support real-time analytics and reporting.
• Utilize Python, SQL, and PL/SQL for data transformation and integration tasks.
• Employ Apache Spark and Databricks to process large-scale data and optimize analytics workflows.
• Implement infrastructure-as-code practices using Terraform, Terragrunt, or CloudFormation for automated, repeatable deployments.
• Integrate CI/CD tools (e.g., Git, GitHub, Linux-based systems) to streamline testing, deployment, and continuous improvement of data systems.
• Integrate generative AI technologies (such as Amazon Bedrock and Chat-GPT) into data processing and analytics pipelines.
• Support machine learning initiatives by facilitating data collection, processing, and storage.
• Mentor and lead junior team members, sharing expertise in AWS data services and engineering best practices.
• Collaborate with data scientists, analysts, and business stakeholders to gather requirements and translate them into effective technical solutions.
• Apply Agile, Scrum, Design Thinking, and Lean Startup methodologies to manage projects and deliver innovative solutions.

Key Skills:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field.
• 5+ years of professional experience in data engineering or a related field, with a strong emphasis on AWS-based data solutions.
• Demonstrable expertise with AWS data services (Lambda, Glue, Lake Formation, Athena, EMR).
• Proficiency in programming with Python, SQL, and PL/SQL.
• Experience with big data frameworks, particularly Apache Spark and Databricks.
• Proven skills in implementing infrastructure-as-code using Terraform, Terragrunt, or CloudFormation.
• Hands-on experience with CI/CD tools and a deep understanding of DevOps practices in Linux environments.
• Familiarity with integrating generative AI tools and machine learning workflows within data engineering projects.
• AWS Professional Certifications (Solutions Architect, DevOps Engineer, etc.) are highly desirable.
• Excellent problem-solving, communication, and leadership skills with a collaborative, agile mindset.

Salary (Rate): undetermined

City: undetermined

Country: United Kingdom

Working Arrangements: undetermined

IR35 Status: undetermined

Seniority Level: Senior

Industry: IT