Senior Data Engineer

Summary: The Senior Data Engineer is responsible for designing and leading scalable data architectures and pipelines to support analytics and business intelligence, with a focus on data optimization and workflow automation in a cloud-based environment. The role requires extensive experience in Python, PySpark, SQL, and AWS services, particularly in the healthcare sector. The position involves collaboration across teams to ensure data quality and system reliability. Candidates should possess strong problem-solving and communication skills, along with a background in agile development.

Key Responsibilities:

• Design and maintain scalable data pipelines and architectures
• Lead data projects and ensure best practices
• Collaborate across teams to meet data needs
• Optimize data systems for analytics and reporting
• Ensure data quality and system reliability in production environments

Key Skills:

• Healthcare experience
• Python
• PySpark
• AWS
• SQL
• 8+ years of IT experience, with 5+ years in big data processing
• Data lakes (Iceberg format), ETL (Informatica), and data quality
• AWS services: S3, Glue, Redshift, Lambda, EMR, Airflow, Postgres
• BASH/Shell scripting
• Experience with healthcare data and leading data teams
• Agile development experience
• Strong problem-solving and communication skills

Salary (Rate): undetermined

City: New York

Country: USA

Working Arrangements: remote

IR35 Status: outside IR35

Seniority Level: Senior

Industry: IT