Lead Data Engineer – Enterprise Platforms Technology/Remote

Lead Data Engineer – Enterprise Platforms Technology/Remote

Posted Today by Apetan Consulting

Apply
Negotiable
Undetermined
Remote
Remote
Apply
Airflow Amazon Web Services (AWS) Apache Airflow Apache Hadoop Artificial Intelligence AWS Elastic MapReduce (EMR) AWS Kinesis AWS Lambda Azure Kubernetes Service Big Data Computer Science Containerisation Data Engineering Data Governance Data Integration Data Lakes Data Modeling Data Processing Data Quality Data Warehousing Generic Programming Kubernetes Machine Learning Management Python (Programming Language) Software Coding Software Development SQL (Programming Language) Stakeholder Management

Job Title: Lead Data Engineer – Enterprise Platforms Technology

Location-Remote

Job Summary:

We are seeking a Lead Data Engineer to design, build, and scale enterprise-grade data platforms and pipelines. This role focuses on leveraging Python, AWS, SQL, and emerging Generative AI technologies to enable advanced analytics, data integration, and intelligent applications. The ideal candidate will lead technical initiatives, mentor teams, and drive innovation in data engineering practices.

Key Responsibilities:

  • Design and develop scalable data pipelines and ETL/ELT processes
  • Build and maintain data lakes, data warehouses, and real-time data systems
  • Lead architecture and implementation of data solutions on AWS
  • Integrate Generative AI capabilities into data platforms and workflows
  • Collaborate with data scientists, analysts, and business stakeholders
  • Ensure data quality, governance, security, and compliance
  • Optimize data processing performance and cost efficiency
  • Define best practices, coding standards, and data engineering frameworks
  • Mentor junior engineers and lead technical reviews
  • Evaluate and adopt new tools, technologies, and methodologies

Required Skills & Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or related field
  • 8–12+ years of experience in data engineering
  • Strong programming skills in Python
  • Advanced SQL skills and experience with large-scale data processing
  • Hands-on experience with AWS services (e.g., S3, Redshift, Glue, Lambda, EMR)
  • Experience building data pipelines using tools like Airflow or similar
  • Solid understanding of data modeling and data warehousing concepts
  • Experience with big data technologies (Spark, Hadoop)
  • Exposure to Generative AI/LLMs and their integration into data workflows

Preferred Qualifications:

  • Experience with streaming technologies (Kafka, Kinesis)
  • Knowledge of containerization and orchestration (Docker, Kubernetes)
  • Familiarity with ML pipelines and MLOps
  • Experience with data governance and catalog tools
  • AWS certifications (e.g., AWS Certified Data Analytics or Solutions Architect)

Key Competencies:

  • Technical leadership and mentoring
  • Strong problem-solving and analytical skills
  • Stakeholder management and communication
  • Innovation and continuous improvement mindset
  • Ownership and accountability
Apply
Similar Jobs
Loading data...