GCP Data Engineer
Posted Today by Whitehall Resources
Negotiable
Inside
Onsite
City of London, London
Summary: The GCP Data Engineer role at Whitehall Resources involves working on-site three days a week for an initial 12-month contract within the Cyber Security domain. The position requires a senior-level professional to leverage cloud technology and engineering skills to provide advanced data capabilities. Key responsibilities include developing data pipelines, optimizing performance, and collaborating with product managers. The role is classified as inside IR35.
Key Responsibilities:
- API Development: Design, develop, and maintain robust and scalable backend systems and APIs.
- Data Ingestion: Develop and maintain data pipelines to extract data from various sources and load it into Google Cloud environments.
- Data Transformation: Implement data transformation processes, including data cleansing, normalization, and aggregation, to ensure data quality and consistency.
- Data Modeling: Develop and maintain data models and schemas to support efficient data storage and retrieval in Google Cloud platforms.
- Data Integration: Integrate data from multiple sources, both on-premises and cloud-based, using Cloud Composer or other relevant tools.
- Data Lakes: Build data lakes using Google Cloud services such as Big Query.
- Performance Optimization: Optimize data pipelines and queries for improved performance and scalability in Google Cloud environments.
- Collaboration: Work with product managers to ensure superior product delivery to drive business value & transformation.
- Documentation: Document data engineering processes, data flows, and system configurations for future reference and knowledge sharing.
Key Skills:
- 8+ years of experience in data engineering, with a strong focus on Google Cloud Platform (GCP)-based solutions.
- Proficiency in the GCP platform, particularly in Data & AI services (e.g., BigQuery, DataProc, Cloud SQL, DataFlow, Pub/Sub, Cloud Data Fusion, Cloud Composer, Python, SQL).
- Designing, developing, and deploying scalable, reliable, and secure cloud-based solutions using GCP services.
- Translating business requirements into technical specifications.
- Proficiently utilizing core GCP services, such as Compute Engine, Kubernetes Engine (GKE), Cloud Storage, Cloud Functions, Cloud SQL, and BigQuery.
- Implementing and managing GCP networking configurations.
- Using IaC tools like Terraform to automate infrastructure provisioning and management.
- Implementing and managing Continuous Integration/Continuous Delivery (CI/CD) pipelines.
- Automating deployment processes and implementing security best practices to protect cloud infrastructure and data.
- Identifying and resolving performance bottlenecks.
Salary (Rate): undetermined
City: City of London
Country: United Kingdom
Working Arrangements: on-site
IR35 Status: inside IR35
Seniority Level: Senior
Industry: IT