Data Engineer

About Us Our mission is to dominate the betting and gaming industry on a global scale and we need the very best Tech talent to help us achieve this. We recently migrated all of our customers onto our very own proprietary platform - so it's an exciting time to join us. With the help of our new platform, we're able to pioneer new products and drive more advanced, creative technologies. The result? Unrivalled experiences for millions of customers worldwide. Betfred's Technology department is driven by innovation, and you'll be at the heart of unlocking our new platform's potential. So, if you want to help shape the future of betting and gaming, then it's to time to join us.

Job Purpose We are on the lookout for a skilled Data Engineer to become an integral part of our Data Engineering Team. You will not only maintain and optimize our data infrastructure but also spearhead its evolution. Built predominantly on AWS, and utilizing technologies like Pyspark and Iceberg, our infrastructure is designed for scalability, robustness, and efficiency. You will be part of developing sophisticated data integrations with various platforms, developing real-time data solutions, improving automation, and enabling crucial business intelligence.

Job Duties

• Design and Build Data Pipelines: Take ownership of designing, developing, and maintaining robust data pipelines and ETL processes using AWS services such as AWS Glue, AWS Lambda, and AWS S3.
• Contribute to Cloud Migration: Actively participate in and support the migration of our existing Data Warehouse from SQL Server to AWS, with a focus on implementing best practices using S3 and Redshift.
• Ensure Data Quality and Integrity: Develop, implement, and maintain data quality checks and validation procedures to ensure the accuracy and reliability of our data.
• Implement Architectural Solutions: Contribute to the design and implementation of data lakehouse architectures and data warehousing solutions.
• Collaborate and Deliver: Work closely with data scientists and analysts to support the deployment and operationalization of machine learning and advanced analytical solutions.
• Maintain and Optimize: Develop and maintain data documentation and operational procedures, and proactively investigate and resolve data quality issues and performance bottlenecks.
• Stay Current and Innovate: Stay up-to-date with the latest data technologies and industry best practices and apply this knowledge to improve our platform.
• Provide Technical Support: Offer technical guidance and support to Junior Data Engineers and other team members when needed.

Knowledge, Skills And Experience

• Strong AWS Proficiency: Solid understanding and practical experience with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR.
• Core Technical Skills: Proven proficiency in Python, SQL, Pipeline Orchestration, PySpark and a deep understanding of data warehousing concepts.
• Problem-Solving: Demonstrated ability to independently diagnose and resolve complex data issues and performance bottlenecks.
• Communication: Ability to effectively communicate technical concepts and project updates to both technical and non-technical stakeholders.
• Experience with Data Architectures: Hands-on experience with data lakehouse architectures and data warehousing solutions.
• Agile Experience: Experience with Agile development methodologies.
• Security and Best Practices: A solid understanding of data security and privacy best practices.

Whats in it for you? Benefits We offer a variety of competitive benefits, some of which vary depending on the role youre recruited to. Some of what you can expect in this role includes: A competitive rate of pay and pension contribution ( £40,000- £55,000) Generous discretionary bonus schemes, incentives and competitions An annual leave entitlement that increases with length of service Access to an online GP 24/7, 365 days a year for you and your immediate family. Employee wellbeing support through our Employee Assistance Programme Enhanced Maternity & Paternity Pay Long Service Recognition Access to a pay day savings scheme, financial coach and up to 40% of your earned wage ahead of payday, through Wagestream.