Remote role :Senior Data Engineer
Summary: Polished Technologies LLC is looking for a Senior Data Engineer with extensive Azure cloud expertise to design and maintain data solutions for Optum. The role involves architecting data pipelines and storage environments while collaborating with various stakeholders to deliver scalable data products. The ideal candidate will have a strong background in data engineering and experience with Azure tools and technologies. This is a remote position for a duration of 6 months or more.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines and architectures on Microsoft Azure.
- Implement data ingestion, transformation, and processing solutions using Azure Synapse Analytics, Azure Data Lake Storage, Azure Data Factory, Azure Databricks, and Power BI.
- Collaborate with software engineers, business analysts, and other stakeholders to refine data requirements and deliver high-quality data solutions.
- Make data accessible and actionable for reporting and analytics teams across the enterprise.
- Produce scalable, replicable code and engineering solutions to automate repetitive data management tasks and workflows.
- Optimize and tune data pipelines for performance, security, and cost-efficiency within Azure environments.
- Manage and optimize data storage solutions, including data lakes and data warehouses leveraging Azure Data Lake and Synapse.
- Maintain high-quality documentation of data definitions, transformations, and processes to support data governance and security.
- Monitor and troubleshoot data pipeline issues using Azure Monitor and related tools, ensuring timely resolution and minimal downtime.
- Communicate effectively with business stakeholders and technical team members.
- Mentor other data engineers and perform code reviews to ensure adherence to best practices and high-quality deliverables.
Key Skills:
- Bachelor's degree in Computer Science, Engineering, or similar field; or equivalent professional work experience.
- 8+ years of experience in software and data engineering roles.
- 6+ years hands-on experience with data analysis, data modeling, ETL (using PySpark notebooks), and designing data warehousing solutions.
- Experience leveraging AI tools in the software development (or product) lifecycle in order to improve quality and efficiency.
- 3+ years of experience with Databricks (Python/Spark), Azure Data Factory, and Azure Data Lake.
- 3+ years of experience with Star Schema and Snowflake Schema data models.
- Demonstrated experience working in Azure Cloud environments.
- Experience with federated data integrations and data virtualization concepts.
- In-depth understanding of formal architecture, design patterns, and best practices for data engineering.
Salary (Rate): undetermined
City: undetermined
Country: undetermined
Working Arrangements: remote
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Remote role :Senior Data Engineer
6 month +
Remote Location
Polished Technologies LLC is seeking an experienced Data Engineer with deep Azure cloud expertise to design, build, and maintain data solutions that drive decision-making for Optum.
The successful candidate will architect and optimize robust data pipelines and storage environments in Azure, leveraging industry best practices for data integration, transformation, and security.
This role will work closely with software engineers, analysts, and business stakeholders to deliver high-quality, scalable data products in complex enterprise environments.
Responsibilities
Design, develop, and maintain scalable data pipelines and architectures on Microsoft Azure.
Implement data ingestion, transformation, and processing solutions using Azure Synapse Analytics, Azure Data Lake Storage, Azure Data Factory, Azure Databricks, and Power BI.
Collaborate with software engineers, business analysts, and other stakeholders to refine data requirements and deliver high-quality data solutions.
Make data accessible and actionable for reporting and analytics teams across the enterprise.
Produce scalable, replicable code and engineering solutions to automate repetitive data management tasks and workflows.
Optimize and tune data pipelines for performance, security, and cost-efficiency within Azure environments.
Manage and optimize data storage solutions, including data lakes and data warehouses leveraging Azure Data Lake and Synapse.
Maintain high-quality documentation of data definitions, transformations, and processes to support data governance and security.
Monitor and troubleshoot data pipeline issues using Azure Monitor and related tools, ensuring timely resolution and minimal downtime.
Communicate effectively with business stakeholders and technical team members.
Mentor other data engineers and perform code reviews to ensure adherence to best practices and high-quality deliverables.
Required Qualifications
Bachelor s degree in Computer Science, Engineering, or similar field; or equivalent professional work experience.
8+ years of experience in software and data engineering roles.
6+ years hands-on experience with data analysis, data modeling, ETL (using PySpark notebooks), and designing data warehousing solutions.
Experience leveraging AI tools in the software development (or product) lifecycle in order to improve quality and efficiency
3+ years of experience with Databricks (Python/Spark), Azure Data Factory, and Azure Data Lake.
3+ years of experience with Star Schema and Snowflake Schema data models.
Demonstrated experience working in Azure Cloud environments.
Experience with federated data integrations and data virtualization concepts.
In-depth understanding of formal architecture, design patterns, and best practices for data engineering.
Preferred Qualifications
Experience with Snowflake.
Experience in the health care industry.
Similar Jobs
Apologies, there are currently no similar jobs available.