Data Platform Engineer

Job Summary

We are seeking an experienced Data Platform Engineer to design, build, and enhance a cloud-native data platform on AWS and Snowflake. The ideal candidate will have strong expertise in data ingestion, transformation, orchestration, metadata management, and data quality, while building scalable and reliable data pipelines that support enterprise analytics and reporting.

Key Responsibilities

• Design and maintain data ingestion pipelines from internal and external sources into AWS S3.
• Develop and manage Airflow DAGs on EKS for data ingestion, transformation, orchestration, and notifications.
• Build and maintain dbt models for data transformation in Snowflake.
• Configure and manage Snowflake integrations, including external stages and environment separation.
• Develop and maintain data quality frameworks and orchestration metadata.
• Integrate with AWS Glue Data Catalog and metadata management platforms.
• Ensure platform reliability, scalability, performance, and cost optimization.
• Collaborate with analytics and reporting teams to support BI and downstream data consumers.
• Implement CI/CD pipelines and automate deployment processes.
• Support containerized workloads and cloud-native platform operations.

Required Skills

• Strong experience with Python and SQL.
• Hands-on experience with Snowflake, Airflow, and dbt (4 - 6 years preferred).
• Experience with AWS services including:
• IAM
• S3
• SNS
• SQS
• API Gateway
• Lambda
• DynamoDB
• EKS
• Experience building and managing S3-based data lakes.
• Knowledge of metadata management, data governance, and data catalogs.
• Experience with CI/CD tools such as GitHub, Azure DevOps, or Octopus.
• Strong Linux administration and scripting skills (Bash, Python, PowerShell).
• Experience with container technologies.

Preferred Qualifications

• Experience with PySpark.
• Knowledge of AWS Glue Catalog, IAM cross-account access, and secure data sharing.
• Experience implementing enterprise data quality frameworks.
• Experience with Kubernetes and EKS-based data workloads.
• Exposure to hybrid cloud and on-premises integrations.
• Experience with hybrid architecture deployments.
• Familiarity with BI tools such as Cognos or Tableau.

Preferred Skills

• Strong analytical and problem-solving abilities.
• Excellent communication and collaboration skills.
• Experience working in Agile development environments.
• Ability to build scalable, secure, and cloud-native data platforms.