Data Engineer
Posted 3 days ago by 1762534058
Negotiable
Outside
Remote
USA
Summary: The Data Engineer role requires expertise in Spark Scala, AWS cloud, and Python, with a strong emphasis on AWS experience. The position involves developing cloud-based solutions, optimizing performance, and ensuring high availability. Candidates must be proficient in Python scripting and familiar with MongoDB and CI/CD practices. This role is primarily remote and classified as outside IR35.
Key Responsibilities:
- Develop and maintain cloud-based solutions using Spark Scala and AWS.
- Write clean, modular, and testable Python code.
- Implement Cloud Batch Development on AWS.
- Design and optimize MongoDB schemas, indexing, and aggregation pipelines.
- Collaborate with leads to suggest design and performance improvements.
- Utilize version control (Git) and Agile methodologies in development processes.
Key Skills:
- Expertise in Python, Scala, and Spark.
- Strong experience with AWS cloud services.
- Proficiency in PySpark and Python scripting.
- Understanding of MongoDB and its design principles.
- Familiarity with CI/CD pipelines and Agile methodologies.
Salary (Rate): undetermined
City: undetermined
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Role: Data Engineer
Location: Woodland Hills, CA/Remote
Duration: 12 months
Job description:
Expertise in Spark scala , AWS cloud and python. AWS Experience is Mandatory.
Strong proficiency in Python scripting & PySpark
Hands-on experience with Cloud Batch Development (AWS)
Understanding of MongoDB, including schema design, indexing, and aggregation pipelines.
Familiarity with AWS / Azure ecosystem
Proficient in writing clean, modular, and testable Python code.
Experience with version control (Git), CI/CD pipelines, and Agile methodologies.
Suggest design & performance improvements to leads to ensuring high availability and performance
Mandatory Skills - Python, Scala, Spark, Aws Cloud