Lead Data Engineer

Summary: The Lead Data Engineer is responsible for designing and leading scalable data architectures and pipelines to support analytics and business intelligence. This role emphasizes data optimization, workflow automation, and ensuring reliable data operations within a cloud-based environment. The position requires extensive experience in IT and data engineering, particularly in healthcare data. The role is remote and focuses on collaboration across teams to meet data needs effectively.

Key Responsibilities:

• Design and maintain scalable data pipelines and architectures
• Lead data projects and ensure best practices
• Collaborate across teams to meet data needs
• Optimize data systems for analytics and reporting
• Ensure data quality and system reliability in production environments

Key Skills:

• 8+ years of IT experience, with 5+ years in Python, PySpark, and SQL for big data processing
• Experience with data lakes (Iceberg format), ETL (Informatica), and data quality
• Proficiency in AWS services: S3, Glue, Redshift, Lambda, EMR, Airflow, Postgres
• BASH/Shell scripting
• Experience with healthcare data and leading data teams
• Agile development experience
• Strong problem-solving and communication skills

Salary (Rate): undetermined

City: undetermined

Country: USA

Working Arrangements: remote

IR35 Status: outside IR35

Seniority Level: undetermined

Industry: IT