Data Engineer 3 (backfill) 100% remote

Data Engineer 3 (backfill) 100% remote

Posted 1 week ago by New York Technology Partners

Negotiable
Undetermined
Remote
Remote

Summary: The Data Engineer 3 role involves managing data operations, including design, quality, and governance, primarily using AWS and Snowflake technologies. The position requires strong technical skills in SQL and Python, along with the ability to communicate complex ideas effectively. The engineer will develop solutions for data issues, automate processes, and collaborate with various teams to enhance data pipeline performance. This is a fully remote position aimed at supporting data-related tasks and improving digital product data quality.

Key Responsibilities:

  • Perform all necessary data related tasks including data design, data quality, data triage, data governance, or data architecture using SQL, Snowflake, and AWS.
  • Develop break/fix solutions and address root causes in data pipeline implementation/code using Python and AWS.
  • Develop scripts and automation tools to better detect and correct data issues.
  • Develop monitoring and alerting capabilities to proactively detect data issues.
  • Work directly on complex application/technical problem identification and resolution, including responding to off-shift and weekend support calls.
  • Identify, investigate, and obtain resolution commitments for platform and data issues to maintain and improve quality and performance of assigned digital product data.
  • Review reports from customers, dealers, industry representatives, and subsidiaries for issue identification.
  • Perform statistical analysis, data triage, and infrastructure problem-solving for issue investigation.
  • Identify root causes, create SageMaker scripts to fix data, and perform break/fix tasks on data pipeline code for issue resolution.
  • Communicate with end users and internal customers to help direct development, debugging, and testing of application software for accuracy, integrity, interoperability, and completeness.
  • Participate in technical sync-ups and meetings with internal teams including US and offshore teams.
  • Liaise with designers, engineers, and support teams to improve data pipeline performance and reliability.
  • Perform other job duties as assigned by Caterpillar management from time to time.

Key Skills:

  • A 4-year degree and/or Master’s degree from an accredited college or university, or equivalent experience.
  • 5-7 years experience in data management, data engineering, or data operations.
  • Experience with development and delivery of microservices using serverless AWS services (S3, RDS, Aurora, DynamoDB, Lambda, SNS, SQS, Kinesis, IAM).
  • 4+ years of experience in a cloud environment (AWS, Snowflake).
  • Strong SQL and Python skills.
  • Verbal and written communication skills with the ability to clearly communicate complex technical ideas.
  • Problem-solving skills, customer service, and interpersonal skills.
  • Ability to work collaboratively in a complex, rapidly changing, and culturally diverse environment.
  • Strong AWS experience, including SageMaker, S3, RDS, CloudWatch and related services (nice to have).
  • Strong Snowflake experience (nice to have).
  • Understanding of logical data domains, primarily Customer & Equipment Domains (nice to have).
  • Strong knowledge of end-to-end data lifecycle across traditional data warehouses, relational databases, operational data stores, business intelligence reporting, and big data analytics (nice to have).
  • Knowledge of data technology products and components for Big Data and Cloud (AWS, Data Lakes, and similar) (nice to have).

Salary (Rate): £45.00 hourly

City: undetermined

Country: undetermined

Working Arrangements: remote

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:

Required Education
A 4-year degree and/or Master s degree from an accredited college or university, or equivalent experience.

Technical Skills (Must-Have)
5 7 years experience in data management, data engineering, or data operations (data design, data quality, metadata, governance, etc.)
Experience with development and delivery of microservices using serverless AWS services (S3, RDS, Aurora, DynamoDB, Lambda, SNS, SQS, Kinesis, IAM)
4+ years of experience in a cloud environment (AWS, Snowflake)
Strong SQL and Python

Soft Skills (Must-Have)
Verbal and written communication skills with ability to clearly communicate complex technical ideas
Problem solving skills, customer service, and interpersonal skills
Ability to work collaboratively in a complex, rapidly changing, and culturally diverse environment

Desired Skills (Nice to Have)
Strong AWS experience, including SageMaker, S3, RDS, CloudWatch and related services
Strong Snowflake experience
Strong SQL and Python development skillset
Understanding of logical data domains, primarily Customer & Equipment Domains
Experience in Data Operations, Tier 2 Support, or comparable Data Engineering Support role
Strong knowledge of end-to-end data lifecycle across traditional data warehouses, relational databases, operational data stores, business intelligence reporting, and big data analytics
Knowledge of data technology products and components for Big Data and Cloud (AWS, Data Lakes, and similar)

Job Responsibilities
Perform all necessary data related tasks including data design, data quality, data triage, data governance, or data architecture SQL, Snowflake, AWS
Develop break/fix solutions and address root causes in data pipeline implementation/code Python, AWS
Develop scripts and automation tools to better detect and correct data issues
Develop monitoring and alerting capabilities to proactively detect data issues
Work directly on complex application/technical problem identification and resolution, including responding to off-shift and weekend support calls
Identify, investigate, and obtain resolution commitments for platform and data issues to maintain and improve quality and performance of assigned digital product data
Issue Identification: Review reports from customers, dealers, industry representatives, and subsidiaries
Issue Investigation: Perform statistical analysis, data triage, and infrastructure problem-solving
Issue Resolution: Identify root causes, create SageMaker scripts to fix data, and perform break/fix tasks on data pipeline code
Communicate with end users and internal customers to help direct development, debugging, and testing of application software for accuracy, integrity, interoperability, and completeness
Participate in technical sync-ups and meetings with internal team including US and offshore teams
Liaise with designers, engineers, and support teams to improve data pipeline performance and reliability
Perform other job duties as assigned by Caterpillar management from time to time