Summary: The role of Data Engineer requires a candidate with SC clearance and a strong grasp of data concepts, including data types, structures, and security principles. The position is fully remote with occasional travel to Central London and involves working with complex data pipelines and various programming languages and frameworks. The successful applicant will need to demonstrate strong analytical and problem-solving skills, particularly in a dynamic Agile environment. The contract duration is an initial 6 months, with a requirement for security clearance prior to appointment.

Key Responsibilities:

• Develop and manage data pipelines using Python/PySpark frameworks.
• Manipulate complex JSON and manage schema structures.
• Ensure data security principles are adhered to, including GDPR compliance.
• Provide support and demonstrate incident triage and handling skills.
• Utilize fundamental Linux system administration knowledge.
• Work within a dynamic Agile environment, utilizing tools like JIRA.
• Integrate and process data using Azure Databricks and Apache Spark.

Key Skills:

• Strong understanding of data concepts, types, and structures.
• Proficiency in Python, SQL, and PySpark.
• Experience with JSON, YAML, and Bash scripting.
• Knowledge of data security principles and handling sensitive datasets.
• Familiarity with Agile methodologies and tools like JIRA.
• Experience with Azure Databricks and Delta Tables.
• Analytical and problem-solving skills.

Salary (Rate): £645

City: London

Country: United Kingdom

Working Arrangements: remote

IR35 Status: inside IR35

Seniority Level: Mid-Level

Industry: IT