Purpose of the Role
To design, build, and optimise data pipelines and integration processes required for migrating an SAS estate (including SAS 9.4 and Viya 3.5 components) to SAS Viya 4 on DPS AWS. The Data Engineer will ensure data integrity, performance, and compliance throughout the migration and operational phases.
Key Responsibilities
Data Pipeline Development:
* Design and implement ETL processes to migrate data from legacy SAS environments to SAS Viya 4.
* Optimise data flows for performance and scalability in a cloud-native architecture.
Integration & Transformation:
* Work with SAS developers to refactor existing jobs and ensure compatibility with Viya 4.
* Implement data transformations and validation rules to maintain accuracy and consistency.
Automation & CI/CD:
* Develop automated workflows for data ingestion and processing using DevOps tools (Jenkins, Git).
* Integrate data pipelines into CI/CD frameworks for continuous delivery.
Data Quality & Governance:
* Ensure compliance with data governance, security, and GDPR standards.
* Implement monitoring and alerting for data quality and pipeline health.
Collaboration:
* Work closely with architects, business analysts, and test teams to align data solutions with migration requirements.
* Support UAT and production readiness activities by providing test data and validation scripts.
Required Skills & Experience
Strong experience in:
* Building and managing ETL pipelines using tools such as SAS Data Integration, Pentaho, or Talend.
* Working with SAS technologies (SAS 9.4, SAS Viya 3.x/4).
Proficiency in:
* SQL, Python, and scripting for data processing.
* Cloud platforms (AWS) and containerisation (Docker, Kubernetes).
Familiarity with:
* Data modelling, performance tuning, and large-scale data migration.
* Agile delivery and DevOps practices.
Desirable Qualifications
* Certifications in SAS, AWS Data Engineering, or equivalent.
* Experience in government or regulated environments.
* Knowledge of data virtualisation tools (e.g., Denodo).