Data Engineer (Python / PySpark / Data pipelines / Big Data)
Posted 1 day ago by GIOS Technology
£45 Per hour
Undetermined
Hybrid
Glasgow, Scotland, United Kingdom
Summary: The Data Engineer role focuses on building and maintaining data architectures and pipelines using Python and PySpark in a hybrid work environment based in Glasgow. The position requires strong technical skills in data processing, performance tuning, and the design of data warehouses and lakes. Candidates should have experience with SQL and a solid understanding of distributed systems. Familiarity with version control systems and AWS is advantageous.
Key Responsibilities:
- Build and maintain robust data architectures and pipelines for data transfer and processing.
- Design and implement data warehouses and data lakes to handle large volumes of data.
- Ensure data security and compliance requirements are met.
- Perform performance tuning, profiling, and resolve production issues in distributed systems.
Key Skills:
- Strong experience with Python and PySpark.
- Proficiency in SQL.
- Knowledge of data architectures, data pipelines, data warehouses, and data lakes.
- Experience with version control systems like Git, GitLab, or Bitbucket.
- Familiarity with AWS.
Salary (Rate): £45.00/hr
City: Glasgow
Country: United Kingdom
Working Arrangements: hybrid
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
We are hiring for Data Engineer (Python / PySpark / Data pipelines / Big Data) Location : Glasgow - Hybrid Strong experience with Python, PySpark, and SQL. Build and maintain robust data architectures and pipelines to ensure durable, complete, and consistent data transfer and processing. Proficiency in Core Java, including Collections, Concurrency, and Memory Management. Design and implement data warehouses and data lakes that can handle large volumes of data and meet all security requirements. A solid background in performance tuning, profiling, and resolving production issues in distributed systems. Experience with version control systems like Git, GitLab, or Bitbucket & AWS is a plus. Key Skills : Data architectures / Data pipelines / data warehouses / data lakes / Python / PySpark