Senior Data Engineer with Cloud Migration | 100% Remote | W2 Profiles

Role: Senior Data Engineer

Location: 100% Remote

Description

We're looking for a Data Engineer to design and maintain data pipelines that move and transform data across AWS and Google Cloud Platform environments, with a strong emphasis on PostgreSQL and Cloud SQL. This role centers on building a pipeline that ingests streaming data from AWS Firehose, stages it in GCS, loads it into Cloud SQL (PostgreSQL), applies data masking, and delivers clean data into BigQuery for analytics.

Key Responsibilities

• Design, build, and maintain ETL/ELT pipelines using Dataflow, Dataproc, and Airflow (Cloud Composer) for orchestration.
• Build and manage data ingestion from AWS Firehose into Google Cloud Storage (GCS).
• Load and manage relational data in Cloud SQL (PostgreSQL), including schema design, performance tuning, and query optimization.
• Implement data masking/anonymization using Cloud DLP to protect sensitive data prior to downstream use.
• Load masked/transformed data into BigQuery for analytics and reporting consumption.
• Use Datastream for change data capture (CDC) and database replication where applicable.
• Monitor pipeline health, troubleshoot failures, and optimize for reliability and cost.
• Collaborate with data analysts, architects, and security/compliance teams to ensure data governance standards are met.

Required Skills & Experience

• Hands-on experience with Google Cloud Platform data services: Dataflow, Dataproc, Cloud Composer/Airflow, BigQuery, Cloud SQL, Datastream.
• Strong PostgreSQL experience — schema design, query optimization, administration.
• Experience with Cloud DLP (or equivalent data masking/anonymization tools).
• Experience working with AWS services, particularly Kinesis Data Firehose, and cross-cloud data movement (AWS GCS).
• Solid understanding of ETL/ELT design patterns, batch and streaming data processing.
• Proficiency in Python and/or SQL for pipeline development.
• Familiarity with data governance, security, and compliance practices (PII handling, masking).

Nice to Have

• Experience with Terraform or other IaC tools for provisioning Google Cloud Platform resources.
• Experience with CI/CD pipelines for data engineering workflows.
• Background in SAP data extraction/integration.