Google Cloud Platform Data Engineer - Remote / Telecommute
Posted 2 weeks ago by 1750936950
Negotiable
Outside
Hybrid
USA
Summary: The Google Cloud Platform Data Engineer will focus on designing, developing, and implementing scalable solutions on the Google Cloud Platform to deliver business value. The role involves managing data pipelines, automating deployments, and ensuring data security and compliance. Collaboration with cross-functional teams is essential to align technical solutions with business objectives. The position offers flexibility in working arrangements, including remote or hybrid options.
Key Responsibilities:
- Design and implement data pipelines using Google Cloud Platform services.
- Oversee containerized deployments of data applications on Kubernetes Engine using Helm.
- Implement CI/CD pipelines using Jenkins for data pipeline deployment and integration.
- Write efficient Python and SQL scripts for data pipelines and ETL/ELT processes.
- Continuously monitor and optimize data workflows for performance and cost-effectiveness.
- Design workflows to integrate data from various sources and orchestrate tasks with Cloud Composer.
- Ensure data security and compliance by implementing IAM policies and encryption.
- Collaborate with application developers, data architects, and business stakeholders.
- Provide technical leadership and ensure alignment between business and technical objectives.
- Set up monitoring, logging, and alerting to ensure visibility into pipeline performance.
Key Skills:
- Experience with Google Cloud Platform services.
- Proficiency in Python and SQL.
- Knowledge of Kubernetes and Helm for container orchestration.
- Experience with CI/CD tools, particularly Jenkins.
- Familiarity with data integration and orchestration tools like Cloud Composer.
- Understanding of data security and compliance best practices.
- Strong collaboration and communication skills.
- Ability to monitor and optimize data workflows.
- Technical leadership capabilities.
Salary (Rate): undetermined
City: undetermined
Country: USA
Working Arrangements: hybrid
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Detailed Description From Employer:
Job Description:
- The Google Cloud Platform Data Engineer will work iteratively on the cloud platform to design, develop and implement scalable, high performance solutions that offer measurable business value to customers.
- Data Solutions: Design and implementation of data pipelines using Google Cloud Platform services
- Manage Deployments with Helm and Kubernetes: Oversee containerized deployments of data applications on Kubernetes Engine using Helm for package management, ensuring efficient orchestration of services.
- Automation with Jenkins: Implement CI/CD pipelines using Jenkins to automate data pipeline deployment, testing, and integration with other services, ensuring quick iterations and deployments.
- Develop and Optimize Pipelines: Write efficient Python and SQL scripts to build data pipelines and ETL/ELT processes.
- Continuously monitor and optimize data workflows for performance and cost-effectiveness.
- Data Integration and Orchestration: Design workflows to integrate data from various sources using Google Cloud Platform services, and orchestrate complex tasks with Cloud Composer (Apache Airflow).
- Security and Compliance: Ensure data security and compliance by implementing IAM policies, encryption, and other security measures in Google Cloud Platform, adhering to best practices for handling sensitive data.
- Collaborate Across Teams: Work closely with application developers, data architects, and business stakeholders to define and deliver robust data-driven solutions.
- Provide technical leadership and ensure alignment between business and technical objectives.
- Monitoring and Logging: Set up monitoring, logging, and alerting using Cloud Monitoring (formerly Stackdriver), Datadog, or other tools to ensure visibility into pipeline performance and quickly identify and resolve issues.