Cloud Data Engineer
Posted 1 week ago by York Solutions, LLC
Negotiable
Undetermined
Hybrid
Remote or Hybrid
Summary: The Cloud Data Engineer role requires a professional with a minimum of 5 years of hands-on experience in cloud data engineering. The position involves designing and implementing scalable data storage solutions, developing data pipelines, and ensuring compliance with data governance and security policies. The engineer will also collaborate with various teams to optimize data systems and support data-related issues. This role offers a flexible working environment with opportunities for remote work.
Key Responsibilities:
- Designing and implementing scalable and secure data storage solutions in the cloud, ensuring optimal performance and accessibility.
- Developing and maintaining robust data pipelines for the ingestion, transformation, and distribution of large datasets.
- Automating data processes and integrating third-party services.
- Utilizing cloud services and tools to automate data workflows and streamline the data engineering process.
- Ensuring compliance with data governance and security policies, including data encryption and access controls.
- Monitoring cloud data systems' performance (Cloud Watch), identifying bottlenecks, and implementing improvements to enhance efficiency.
- Conducting data quality checks and implementing measures to ensure data accuracy and integrity.
- Optimizing data retrieval and developing APIs for data consumption by various enterprise consumers.
- Providing technical expertise and support for data-related issues, including troubleshooting and resolving data pipeline failures.
- Collaborating with IT and security teams to plan and execute disaster recovery strategies for cloud-based data systems.
- Documenting data engineering processes, creating data flow diagrams, and maintaining metadata for data lineage and cataloging.
- Collaborating with architects, analysts, and other engineers to support data modeling, analysis, and reporting needs.
- Staying current with emerging cloud technologies and data engineering practices to recommend and adopt innovations that improve data systems.
Key Skills:
- Cloud Platforms: AWS (Kubernetes), Google Cloud, Microsoft Azure.
- Data Storage: Data lakes, data warehouses, cloud storage service (S3).
- Data Pipelines: Developing and maintaining data pipelines for ETL processes leveraging GitHub Actions for promotions of code assets.
- Programming Languages: Python, Java, Scala.
- Data Management Systems: SQL, NoSQL, Hadoop, Postgress.
- Data Security and Governance: Understanding of data security best practices and compliance regulations.
- Problem-Solving and Analytical Skills: Ability to troubleshoot issues and analyze data to identify patterns and trends.
Salary (Rate): £60 hourly
City: undetermined
Country: undetermined
Working Arrangements: hybrid
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Position Overview: Cloud Data Engineer: - Mandatory 5 Years of Hands on Experience
Key Skills:
? Cloud Platforms: AWS ( Kubernetes), Google Cloud, Microsoft Azure.
? Data Storage: Data lakes, data warehouses, cloud storage service (S3)
? Data Pipelines: Developing and maintaining data pipelines for ETL processes leveraging GitHub Actions for promotions of code assets.
? Programming Languages: Python, Java, Scala
? Data Management Systems: SQL, NoSQL, Hadoop, Postgress
? Data Security and Governance: Understanding of data security best practices and compliance regulations.
? Problem-Solving and Analytical Skills: Ability to troubleshoot issues and analyze data to identify patterns and trends.
Responsibilities
? Designing and implementing scalable and secure data storage solutions in the cloud, ensuring optimal performance and accessibility.
? Developing and maintaining robust data pipelines for the ingestion, transformation, and distribution of large datasets.
? Automating data processes and integrating third-party services
? Utilizing cloud services and tools to automate data workflows and streamline the data engineering process.
? Ensuring compliance with data governance and security policies, including data encryption and access controls.
? Monitoring cloud data systems' performance (Cloud Watch), identifying bottlenecks, and implementing improvements to enhance efficiency.
? Conducting data quality checks and implementing measures to ensure data accuracy and integrity.
? Optimizing data retrieval and developing APIs for data consumption by various enterprise consumers.
? Providing technical expertise and support for data-related issues, including troubleshooting and resolving data pipeline failures.
? Collaborating with IT and security teams to plan and execute disaster recovery strategies for cloud-based data systems.
? Documenting data engineering processes, creating data flow diagrams, and maintaining metadata for data lineage and cataloging.
? Collaborating with architects, analysts, and other engineers to support data modeling, analysis, and reporting needs.
? Staying current with emerging cloud technologies and data engineering practices to recommend and adopt innovations that improve data systems.
Benefits:
York Solutions Offers a generous benefits package for eligible full-time employees:
- BCBS Medical with 3 Plans to choose from (PPO and High deductible PPO plans with Health Savings Program)
- Delta Dental plan with 2 free cleanings and insurance discounts
- Eye Med Vision with annual check-ups and discounts on lens
- Life and Accidental Death Insurance paid by company
- John Hancock 401(k) Retirement Plan with discretionary company match
- Voluntary Insurance programs such as: Hospital Indemnity, Identity Protection, Legal Insurance, Long Term Care, and Pet Insurance.
- Flexible work environment with some remote working opportunities
- Strong fun and teamwork environment
- Learning, development, and career growth