Data Engineer

Data Engineer

Posted 1 week ago by FDM Group on Linkedin

£75,000 Per year
Undetermined
Remote
London Area, United Kingdom

Summary: FDM is seeking a SAS Data Engineer for a fully remote, initially 6-month contract role focused on migrating SAS scripts to a cloud environment, particularly using Snowflake, Python, and SparkSQL. The position requires a strong foundation in data quality and the ability to resolve discrepancies to ensure effective data processes. The role involves developing ETL scripts, conducting data quality checks, and collaborating with cross-functional teams. Opportunities for career coaching and upskilling are also provided.

Key Responsibilities:

  • Develop and optimise ETL scripts using Snowflake, primarily utilising SQL
  • Ensure the quality and integrity of the data produced by conducting rigorous automated data checks
  • Investigate and resolve any data discrepancies that arise during the ETL process
  • Design and implement orchestration workflows for Snowflake and other scripts using Apache Airflow
  • Implement logging and monitor and notifications for any failures or issues
  • Work closely with cross-functional teams, exemplifying best practices and sharing knowledge
  • Write and deploy SparkSQL scripts as part of the data transformation processes, running on Amazon EMR

Key Skills:

  • Strong Snowflake SQL skills, with the ability to write complex queries and optimise performance
  • Strong Python for scripting and automation tasks
  • Experience with Apache Airflow for data pipeline orchestration
  • Strong problem-solving skills with attention to detail, particularly in data validation and quality assurance
  • Competent in reading and interpreting SAS scripts
  • Able to collaborate with others and communicate effectively best practices

Salary (Rate): £75,000.00

City: London

Country: United Kingdom

Working Arrangements: remote

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT