Data engineer

Data engineer

Posted 1 day ago by Analytics Solutions

Apply
Negotiable
Undetermined
Remote
Remote
Apply
Automation BigQuery Cloud Computing Cloud Technology Continuous Integration and Continuous Delivery Data Architecture Data Processing Device Tracking Software Dynamic Queries Generic Programming Git (Version Control System) Google Cloud Google Cloud Platform (GCP) Hardening Performance Tuning Python (Programming Language) Scripting Software Development SQL (Programming Language) Teradata SQL Workflows

Summary: The Data Engineer role focuses on developing and maintaining data architecture and workflows, with a strong emphasis on Python orchestration and Google Cloud Platform services. The position requires proficiency in BigQuery and related technologies, ensuring efficient data processing and integration. This is a long-term contract position that allows for remote work. Candidates should have a solid understanding of CI/CD practices and performance tuning in data environments.

Key Responsibilities:

  • Design and build data architecture and workflows, ensuring adherence to standards.
  • Implement Python-heavy orchestration for workflow execution and status tracking.
  • Develop and optimize BigQuery scripts, stored procedures, and dynamic SQL.
  • Automate API/service development and deployment on Google Cloud Platform.

Key Skills:

  • Proficiency in BigQuery SQL and scripting.
  • Strong Python programming skills for orchestration and error handling.
  • Experience with Git and CI/CD practices.
  • Familiarity with Google Cloud Platform services.
  • Knowledge of metadata/config-driven design and testing strategies.
  • Understanding of Teradata SQL concepts.

Salary (Rate): £37.50 hourly

City: undetermined

Country: undetermined

Working Arrangements: remote

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:

Job Title: Data engineer

Location: Remote

Duration: Long Term Contract

Proficiency:

  • Architecture(for Lead) + hands-on build; can understand & build campaigns following the standards, CI/CD, monitoring.
  • Python-heavy orchestration (workflow execution, status tracking, retries), integration patterns, production hardening.
  • BigQuery scripting, stored procedures, dynamic SQL, performance tuning, partitioning/clustering strategies.
  • API/service development, security, deployment automation on Google Cloud Platform.

Technology Expectations

Core(Must have)

  • BigQuery SQL & scripting (procedures, dynamic SQL, temp/operational tables, performance)
  • Python (orchestration, config-driven execution, logging/error handling)
  • Git + CI/CD (code review, automated deployments, environment promotion)
  • Onshore-focused(Lead - must have, developers - good to have)
  • Google Cloud Platform services (as applicable): Cloud Composer/Airflow, Cloud Run, Pub/Sub, GCS, Secret Manager, Cloud Logging/Monitoring, IAM
  • Metadata/config-driven design (YAML/config tables), execution-status tracking, testing strategy.
  • Teradata SQL (BTEQ concepts, Teradata functions/qualify/volatile tables equivalents)
  • BigQuery translation patterns + validation/reconciliation

Apply
Similar Jobs
Loading data...