Google Cloud Platform Data Engineer (Google Cloud Platform)
Posted Today by 1763793623
Negotiable
Outside
Remote
USA
Summary: The Google Cloud Platform Data Engineer role focuses on designing and implementing data pipelines using Google Cloud services, managing workflows, and ensuring data quality. The position requires strong programming skills in Python and SQL, as well as experience with real-time data processing and NoSQL databases. The role is remote and emphasizes collaboration and communication with stakeholders. Candidates should have a minimum of three years of hands-on experience in relevant technologies.
Key Responsibilities:
- Design and implement robust data pipelines using Google Cloud Platform services such as BigQuery, Cloud Storage, and Pub/Sub.
- Develop and manage workflows using Cloud Composer (Apache Airflow) for efficient scheduling and orchestration.
- Write clean, efficient, and scalable code in Python, leveraging advanced programming techniques.
- Craft complex SQL queries in BigQuery, including window functions, CTEs, and performance tuning strategies.
- Build and maintain real-time data processing systems using Apache Kafka.
- Model and manage NoSQL databases, particularly MongoDB, with a focus on scalable schema design.
- Utilize Shell scripting and perform Linux system administration tasks to support data infrastructure.
- Conduct data profiling and implement validation techniques to ensure data quality and integrity.
- Develop and maintain API integration scripts for seamless service automation and data exchange.
- Troubleshoot and resolve data-related issues with strong analytical and problem-solving skills.
- Create and maintain data flow diagrams to clearly communicate architecture and pipeline logic to stakeholders.
Key Skills:
- Minimum 3 years of hands-on experience with Google Cloud Platform services.
- Proficiency in Python programming and SQL.
- Experience with Cloud Composer (Apache Airflow) and real-time data processing using Apache Kafka.
- Knowledge of NoSQL databases, particularly MongoDB.
- Familiarity with Shell scripting and Linux system administration.
- Strong analytical and problem-solving skills.
- Ability to create data flow diagrams and communicate effectively with stakeholders.
Salary (Rate): undetermined
City: undetermined
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Job Title: Google Cloud Platform Data Engineer Location: REMOTE Duration: Long Term
Required minimum 3 years of proven hands-on experience in the following:
Design and implement robust data pipelines using Google Cloud Platform (Google Cloud Platform) services such as BigQuery, Cloud Storage, and Pub/Sub.
Develop and manage workflows using Cloud Composer (Apache Airflow) for efficient scheduling and orchestration.
Write clean, efficient, and scalable code in Python, leveraging advanced programming techniques.
Craft complex SQL queries in BigQuery, including window functions, CTEs, and performance tuning strategies.
Build and maintain real-time data processing systems using Apache Kafka.
Model and manage NoSQL databases, particularly MongoDB, with a focus on scalable schema design.
Utilize Shell scripting and perform Linux system administration tasks to support data infrastructure.
Conduct data profiling and implement validation techniques to ensure data quality and integrity.
Develop and maintain API integration scripts for seamless service automation and data exchange.
Troubleshoot and resolve data-related issues with strong analytical and problem-solving skills.
Create and maintain data flow diagrams to clearly communicate architecture and pipeline logic to stakeholders.