Senior Data Engineer (Python, PySpark, Snowflake, Airflow, AWS Services) )

Senior Data Engineer (Python, PySpark, Snowflake, Airflow, AWS Services) )

Posted 2 days ago by Unitech Consultancy Services Ltd

Apply
Negotiable
Undetermined
Undetermined
Edinburgh/London, UK
Apply
Apache Spark AWS Elastic MapReduce (EMR) Balancing (Ledger/Billing) Cloud Computing Cloud Technology Continuous Integration and Continuous Delivery Data Engineering Data Management Data Mart Datamodel Data Modeling Data Processing Data Warehousing Extract Transform Load (ETL) Financial Services Management Workflows

Summary: We are seeking a Senior Data Engineer with extensive experience to design, build, and optimize scalable data solutions, focusing on delivering robust data models and efficient pipelines. The ideal candidate will possess strong technical expertise in cloud-based data engineering and experience in financial services, particularly with NatWest. This role requires both independent work and collaboration within a team to enable actionable business insights.

Key Responsibilities:

  • Design and implement efficient data models to support reporting and analytics.
  • Build and maintain ETL pipelines and develop scalable data marts.
  • Develop and optimize data processing solutions using Python, Spark, and PySpark.
  • Orchestrate workflows with Apache Airflow.
  • Work with Snowflake to manage and optimize data warehouse solutions.
  • Leverage AWS services (EMR, S3, Lakehouse) to build scalable cloud data platforms.
  • Apply data quality and governance practices to ensure reliability of data.
  • Contribute to CI/CD workflows, preferably using GitLab.
  • Deliver solutions within agreed timelines, balancing quality and speed of execution.

Key Skills:

  • 7-8+ years of professional experience as a Data Engineer.
  • Strong background in data modelling and data mart development.
  • Hands-on expertise in Python, Spark, and PySpark.
  • Strong experience with Airflow for orchestration.
  • Proven track record with Snowflake (data warehouse, advanced features).
  • Knowledge of GitLab CI/CD (preferred).
  • Strong ETL development skills and experience in large-scale data projects.
  • Cloud expertise with AWS EMR, S3, and Lakehouse.
  • Knowledge of data quality and data management frameworks (desirable).
  • Previous Banking experience will be considered highly beneficial.
  • Ability to work independently while contributing within a team environment.
  • Comfortable delivering under tight deadlines.

Salary (Rate): £475 yearly

City: Edinburgh

Country: UK

Working Arrangements: undetermined

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:

We are looking for an experienced Senior Data Engineer to join our team and help design, build, and optimize scalable data solutions. You will be responsible for delivering robust data models, efficient pipelines, and high-quality data marts that enable actionable insights for the business.

The ideal candidate will have deep technical expertise, a strong background in cloud-based data engineering, and the ability to work independently while collaborating within a wider team. Experience working in financial services, particularly with NatWest, will be highly advantageous.

Key Responsibilities

  • Design and implement efficient data models to support reporting and analytics.

  • Build and maintain ETL pipelines and develop scalable data marts.

  • Develop and optimize data processing solutions using Python, Spark, and PySpark.

  • Orchestrate workflows with Apache Airflow.

  • Work with Snowflake to manage and optimize data warehouse solutions.

  • Leverage AWS services (EMR, S3, Lakehouse) to build scalable cloud data platforms.

  • Apply data quality and governance practices to ensure reliability of data.

  • Contribute to CI/CD workflows, preferably using GitLab.

  • Deliver solutions within agreed timelines, balancing quality and speed of execution.

Required Skills & Experience

  • 7-8+ years of professional experience as a Data Engineer.

  • Strong background in data modelling and data mart development.

  • Hands-on expertise in Python, Spark, and PySpark.

  • Strong experience with Airflow for orchestration.

  • Proven track record with Snowflake (data warehouse, advanced features).

  • Knowledge of GitLab CI/CD (preferred).

  • Strong ETL development skills and experience in large-scale data projects.

  • Cloud expertise with AWS EMR, S3, and Lakehouse.

  • Knowledge of data quality and data management frameworks (desirable).

  • Previous Banking experience will be considered highly beneficial.

  • Ability to work independently while contributing within a team environment.

  • Comfortable delivering under tight deadlines.

Apply
Similar Jobs
Loading data...