Senior AWS Technical Lead
Posted Today by Trust In SODA
£420 Per day
Inside
Onsite
Greater London, England, United Kingdom
Summary: The Senior AWS Technical Lead role in Glasgow involves building and maintaining data systems, including pipelines, warehouses, and lakes, to ensure data accuracy and security. The position requires strong hands-on experience in data engineering and the ability to lead projects, with a focus on AWS technologies and Databricks. Candidates will engage with stakeholders to translate business requirements into technical designs and drive improvements in existing processes. The role is expected to last for 12 months and requires a mix of technical and leadership skills.
Key Responsibilities:
- Build and maintain data pipelines for accurate and secure data transfer and processing.
- Design and implement data warehouses and lakes adhering to security measures.
- Develop processing and analysis algorithms for complex data.
- Collaborate with data scientists to deploy machine learning models.
- Lead projects with strong hands-on data engineering experience.
- Engage with stakeholders to understand and translate business requirements into technical designs.
- Proactively analyze existing processes and suggest improvements.
Key Skills:
- Strong SQL and programming skills (Python/Pyspark preferred).
- Experience with AWS services (S3, Lambda, Glue, etc.).
- Hands-on experience with modern orchestration tools (e.g., Airflow, Step Functions).
- Strong experience in Databricks development/migration.
- Knowledge of Apache Spark, Databricks Delta Lake, Unity Catalog, and MLflow.
- Ability to design, implement, and optimize complex data pipelines independently.
- UI development experience along with AWS CloudFormation.
Salary (Rate): £420.00/daily
City: Glasgow
Country: United Kingdom
Working Arrangements: on-site
IR35 Status: inside IR35
Seniority Level: Senior
Industry: IT
Senior AWS Technical Lead
Location: Glasgow – 2/3 Days on site
Duration: 12 Months
Rate: £350 – £420
Inside IR35
Role Description:
To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure. Candidates working for this role will either contribute to strategic deliverables by building data engineering pipelines using the AWS technology stack, or support platform modernization efforts by migrating existing AWS components to Databricks, depending on their assessed skill sets. Should lead the project: Strong hands-on data engineering experience delivering production-grade solutions. Ability to design, implement and optimise complex data pipelines independently. Strong SQL and programming (Python/Pyspark preferred); capable of building reusable and scalable code. Experience with modern orchestration tools (e.g. Airflow, Step Functions, etc.). Hands-on experience on AWS cloud platform with skillsets listed below Ability to proactively analyse existing processes, suggest improvements, and drive technical solutions end-to-end. Comfortable engaging with stakeholders, understanding business requirements, and translating them into technical design. Strong Experience in Databricks development/migration. (this is for Databricks engineer) Experience with Apache Spark, Databricks Delta Lake, Unity Catalog, and MLflow.( this is for Databricks engineer)
Accountabilities
- Build and maintenance of data pipelines that enable the transfer and processing of durable, complete and consistent data.
- Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures.
- Development of processing and analysis algorithms fit for the intended data complexity and volumes.
- Collaboration with data scientist to build and deploy machine learning models.
Hands-on Skills:
- Data Engineering Coding background and Hands on experience on below AWS services.
- S3, Lambda, Glue, Step Function, Athena, Sagemaker, VPC, ECS, IAM, KMS etc.
- CloudFormation
- Python
- Unit Testing
- Gitlab
- PySpark
- AI/ML knowledge (Good to have)
- **UI development expereince as must along with AWS Cloudformation.**