Senior Data Engineer with DBT, Databricks, Spark & SQL :: 100 Remote :: W2/1099
Posted 4 days ago by Trebecon LLC
Negotiable
Undetermined
Remote
Remote
Summary: The Senior Data Engineer role focuses on leveraging extensive experience in data engineering to design and implement scalable data solutions. The position requires proficiency in tools such as DBT, Databricks, and Apache Spark, with a strong emphasis on cloud technologies. This is a fully remote position, allowing for flexibility in work arrangements. Candidates should possess a robust background in SQL and data pipeline development.
Key Responsibilities:
- Design and implement scalable ETL/ELT pipelines using Spark and cloud-native technologies.
- Utilize Apache Spark/PySpark for large-scale distributed data processing.
- Transform and model data using DBT (Data Build Tool).
- Develop data engineering solutions using Databricks.
- Optimize SQL queries and performance tuning for data models.
- Work with large datasets and manage batch processing and real-time/streaming data pipelines.
- Collaborate with teams using Git, CI/CD pipelines, and Agile/Scrum methodologies.
- Ensure a strong understanding of data warehouse, lakehouse, and data lake architectures.
- Apply problem-solving and analytical skills to address data challenges.
Key Skills:
- 10+ years of professional experience as a Data Engineer.
- Hands-on experience with Apache Spark / PySpark.
- Strong experience with DBT for data transformation and modeling.
- Experience with Databricks for data engineering and pipeline development.
- Proficiency in SQL, including query optimization and data modeling.
- Experience building scalable ETL/ELT pipelines.
- Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.
- Understanding of data warehouse, lakehouse, and data lake architectures.
- Experience with large datasets and real-time/streaming data pipelines.
- Knowledge of Delta Lake and Spark optimization techniques is preferred.
- Experience with Git, CI/CD pipelines, and Agile/Scrum methodologies.
- Excellent problem-solving, analytical, and communication skills.
Salary (Rate): undetermined
City: undetermined
Country: undetermined
Working Arrangements: remote
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Position: Senior Data Engineer
Location: 100% Remote
Required Skills:
SQL, DBT, Databricks, Apache Spark, AWS
Required Skills & Qualifications:
- 10+ years of professional experience as a Data Engineer.
- Strong hands-on experience with Apache Spark / PySpark for large-scale distributed data processing.
- Strong experience with DBT (Data Build Tool) for data transformation and modeling.
- Hands-on experience with Databricks for data engineering, Spark-based processing, and pipeline development.
- Proficiency in SQL including query optimization, performance tuning, data modeling, and stored procedures.
- Experience building scalable ETL/ELT pipelines using Spark and cloud-native technologies.
- Experience with at least one cloud platform such as AWS, Azure, or Google Cloud Platform (AWS preferred).
- Strong understanding of data warehouse, lakehouse, and data lake architectures.
- Experience working with large datasets, batch processing, and real-time/streaming data pipelines.
- Knowledge of Delta Lake, Spark optimization techniques, and distributed computing concepts is preferred.
- Experience with Git, CI/CD pipelines, and Agile/Scrum methodologies.
- Excellent problem-solving, analytical, and communication skills.