Data Engineer

About the Role

We are seeking a highly skilled Data Engineer to join our consulting team for a remote engagement with a client headquartered in Boston, MA. If Interested please share your updated resume in word format. Thank you

Key Responsibilities

• Design, build, and optimize big data ingestion pipelines to handle large volumes of structured and unstructured data.
• Develop and maintain real-time streaming solutions using Kafka and Spark Streaming.
• Collaborate with data architects and business stakeholders to define data integration strategies.
• Implement scalable solutions on Azure cloud services and integrate with Snowflake for analytics and reporting.
• Ensure data quality, reliability, and performance across distributed systems.
• Troubleshoot and optimize existing pipelines to improve efficiency and reduce latency.
• Document technical designs, workflows, and best practices for long-term maintainability.
• Implement ELT ingestion using Fivetran, integrating APIs with Python scripting to land data into Snowflake.

Required Skills & Qualifications

• Experience with Fivetran ELT, API integration, and Python scripting for Snowflake ingestion.
• 12+ years of professional experience as a Data Engineer or similar role.
• Strong expertise in Apache Spark (batch and streaming) and Apache Kafka.
• Proven experience with large-scale data ingestion and processing.
• Hands-on experience with Azure Data Services (e.g., Data Lake, Data Factory, Synapse).
• Proficiency in SQL and experience with Snowflake for data warehousing.
• Solid understanding of distributed systems and scalable architectures.
• Strong problem-solving skills and ability to work independently in a remote environment.
• Expertise in Python and Machine Learning for building and optimizing data pipelines.

Preferred Qualifications

• Experience with Fivetran ELT, API integration, and Python scripting for Snowflake ingestion.
• Experience with performance tuning of big data pipelines.
• Familiarity with CI/CD pipelines and DevOps practices for data engineering.
• Exposure to data governance and security best practices.