What you'll be doing
- Designing, building and maintaining repeatable ETL/ELT pipelines that load data into strategic data platforms
- Writing well-tested Python and PySpark code, using Git, CI/CD and agreed engineering standards
- Profiling, cleaning and transforming data, spotting anomalies and resolving quality issues
- Optimising the performance of queries and pipelines
- Developing internal Data Engineering packages and rolling out process improvements
- Working with technical and non-technical stakeholders to turn requirements into working solutions
- Keeping data and processes compliant with departmental policy and legal requirements
- Contributing to Civil Service Data Engineering communities
What you'll need
- Strong practical experience with Python and SQL for data engineering, including preparing and transforming data reproducibly at scale
- Experience designing, coding, testing and documenting moderate-to-complex programs or scripts
- Confident use of Git, code review, testing and documentation
- A proactive approach to investigating, fixing and preventing technical issues
- Clear communication with technical and non-technical colleagues
- Ability to work independently and as part of a team
Tech stack
- Python
- PySpark
- SQL
- Cloudera Data Platform
- Google Cloud Platform
- Git
- CI/CD
Why apply?
- Meaningful public sector work with a national impact
- Flexible working and a strong work / life balance
- Excellent training / progression
- Room to learn new tools and technologies through structured development pathways
IR35 Status:
Not specified
Industry:
Data & Analytics
Seniority Level:
Mid-Level