AI Data Engineer

Detailed Description From Employer:

On behalf of UKRI we are looking for an AI Data Engineer (Inside IR35) for an initial 3 month contract, with potential to extend. Remote working with very occasional travel to Swindon.

UK Research and Innovation (UKRI) is the national funding agency investing in science and research in the UK.

UKRI invests £8 billion of taxpayers' money each year into research and innovation and the people who make it happen. They work across a huge range of fields - from biodiversity conservation to quantum computing, and from space telescopes to innovative health care. They give everyone the opportunity to contribute and to benefit, bringing together people and organisations nationally and globally to create, develop and deploy new ideas and technologies.

The Data Engineer will be responsible for the requirements analysis and design of solutions for data platforms, ETL, integration and analysis solutions. Our toolset includes AWS hosted databases (postgres, S3 based data lake, MySQL, Athena), integration services (AWS API gateway, lambda functions), ETL services (AWS Glue, Step functions, AWS Batch), infrastructure management tools such as Terraform and extensive use of SQL and Python.

As an AI Data Engineer your main responsibilities will be:

Infrastructure Management:

.

. Set up and manage data infrastructure, including clusters, servers, and cloud-based resources using Terraform.
. Monitor and optimize system performance, troubleshoot issues, and ensure system availability.

Data Architecture and Design:

.

. Design and implement scalable and efficient data pipelines, databases, and data warehouses.
. Collaborate with data visualization and analysts' teams to understand data requirements and translate them into technical specifications.

Data Processing:

.

. Develop and maintain ETL (Extract, Transform, Load) processes for ingesting data from various sources into the data infrastructure.
. Optimize data processing and storage for performance and cost-effectiveness.

AI:

.

. Build scalable inference pipelines and AI APIs.
. Implement MLOps workflows for model versioning, monitoring, and retraining.
. Collaborate with data scientists to productionize machine learning solutions.
. Evaluate and optimize AI model performance and operational efficiency.

Database Management:

.

. Manage and maintain databases, ensuring data integrity, security, and availability.
. Implement database schema changes and optimizations as needed.

Collaboration:

.

. Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to meet data requirements.
. Communicate effectively with stakeholders to gather requirements and provide updates on data engineering projects.

Essential:

. Comfortable working in an Agile rapidly changing environment.
. Data Engineering experience, with strong Python skills.
. Exposure, knowledge and/or experience of working with AI technologies.
. Experience with infrastructure deployment tools such as Terraform, CDK or cloud formation.
. Experience developing API based data integration.
. Excellent analytic skills associated with working on structured and unstructured datasets.
. Excellent SQL experience on various platforms (SQL, PostgreSQL, PL/SQL etc).
. Experience of several of MySQL, Oracle, SQL, Postgres, RDS, Aurora, Athena or other similar large scale database technologies.

Desirable:

. Experience working with AI frameworks such as LangChain/LlamaIndex.
. Experience with Vector databases (Pinecone, Weaviate, FAISS).

Please be aware that this role can only be worked within the UK and not Overseas.

Disability Confident

As a member of the Disability Confident Scheme, UKRI guarantees to interview all candidates who have a disability and who meet all the essential criteria for the vacancy. In cases where we have a high volume of candidates who have a disability who meet all the essential criteria, we will interview the best candidates from within that group. This scheme encourages candidates with a disability and/or neurodivergence to apply. In exceptional circumstances, we may also need to apply the desirable criteria in our shortlisting process which may include holding active security clearance.

In applying for this role, you acknowledge the following "this role falls in scope of the Off Payroll Working in the Public Sector legislation. Any rates of payment quoted will reflect the gross rate per day for the assignment and will be subject to appropriate taxes and statutory costs. As such the payment to the intermediary and your income resulting from this contract will be different".