Summary: The Data Engineer role requires an individual with active SC clearance to work remotely for a duration of 4 months, with potential extensions. The position demands a strong understanding of data concepts, complex JSON manipulation, and experience with data pipelines using Python/PySpark frameworks. The candidate should also possess knowledge of data security principles and experience in a dynamic Agile environment.

Key Responsibilities:

• Develop and manage data pipelines using custom Python/PySpark frameworks.
• Manipulate complex JSON data and manage schema structures.
• Ensure data security and compliance with GDPR for sensitive datasets.
• Provide support and incident triage in a data environment.
• Utilize fundamental Linux system administration skills.
• Work collaboratively in an Agile environment using tools like JIRA.

Key Skills:

• Strong understanding of data concepts, types, structures, and schemas.
• Proficiency in Python programming and experience with PySpark.
• Knowledge of data security principles and handling sensitive data.
• Experience with JSON, SQL, and Bash scripting.
• Familiarity with Azure Databricks and Apache Spark technologies.
• Ability to work in a dynamic Agile environment.

Salary (Rate): 640

City: City of London

Country: United Kingdom

Working Arrangements: remote

IR35 Status: inside IR35

Seniority Level: Mid-Level

Industry: IT