Data Engineer [Data, ETL/ELT, cloud, Shell/Bash, Data Modelling]

Role Description

We are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will play a crucial role in designing, building, and maintaining our data infrastructure and systems. You will collaborate with cross-functional teams to ensure efficient and reliable data pipelines, implement data integration solutions, and support data-driven initiatives. This position requires strong technical expertise, problem-solving skills, and a deep understanding of data engineering principles.

Roles & Responsibilities

• Design and Develop Data Infrastructure: Architect, build, and optimize scalable data infrastructure, including data warehouses, data lakes, and ETL/ELT processes.
• Data Integration: Implement efficient and reliable data integration pipelines, ensuring the smooth flow of data from various sources to target systems.
• Data Modelling and Schema Design: Design and implement data models and schemas that support business requirements and optimize data retrieval and analysis.
• Data Quality and Governance: Develop and implement data quality checks, data validation processes, and data governance practices to maintain data integrity and accuracy.
• Performance Optimization: Identify performance bottlenecks, optimize data processing and storage systems, and improve overall data pipeline efficiency.
• Data Security: Ensure data privacy and security by implementing appropriate access controls, encryption techniques, and data protection measures.
• Collaboration: Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand their data needs and provide technical support.
• Documentation and Monitoring: Create and maintain technical documentation, data dictionaries, and monitoring systems to track data pipeline performance and identify issues.
• Troubleshooting and Issue Resolution: Investigate and resolve data-related issues, including data quality issues, system failures, and performance degradation.
• Data Architecture and Technology Evaluation: Stay updated with industry trends and emerging technologies in data engineering and evaluate their potential impact and suitability for the organization.

Qualifications - External/Internal

What you will need to succeed in the role: (Minimum Qualification and Skills Required)

• 5+ years of experience in data engineering, data integration, or related roles.
• Strong proficiency in database management/development and working experience with relational databases, data modelling, and schema design.
• Proficiency in data integration tools, ETL/ELT frameworks, and data pipeline orchestration tools.
• Strong programming skills in languages such as Python/Java.
• Understanding of data warehousing concepts, dimensional modelling, and data governance practices.
• Knowledge of data security and privacy principles, as well as experience implementing data security measures.
• Working experience with cloud platforms (eg, AWS, Azure, GCP) and their data services (eg, AWS Glue, Azure Data Factory, GCP Dataflow).
• Having knowledge of Docker & Kubernetes will be added advantage
• Have good understanding on Shell/Bash Scripting
• Good analytical skills to present and understand statistical trends and forecast
• Excellent problem-solving skills, attention to detail, and the ability to work effectively in a team environment