Lead Data Engineer
Posted 1 week ago by 1754719890
Negotiable
Outside
Remote
USA
Summary: The role of Sr/Lead Data Engineer at KPI Partners involves developing and implementing data pipelines using PySpark, SQL, and Python, while collaborating with cross-functional teams to create data solutions. This position is fully remote and requires a strong background in ETL and data warehousing fundamentals. The ideal candidate will have over 8 years of experience in data engineering and a proactive approach to problem-solving in a dynamic environment. Additional skills in Azure services and data analytics tools are beneficial but not mandatory.
Key Responsibilities:
- Develop and implement data pipelines using PySpark.
- Collaborate with cross-functional teams to translate business requirements into data solutions.
- Utilize SQL for data management and analysis.
- Enhance engineering expertise through industry-leading practices and tools.
- Work independently in a fast-paced environment.
Key Skills:
- 8+ years of experience in data engineering.
- Strong proficiency in PySpark, SQL, and Python.
- Experience with ETL and data warehousing fundamentals.
- Ability to work collaboratively with machine learning engineers and product managers.
- Good to have knowledge of Azure services and data analytics tools.
Salary (Rate): undetermined
City: undetermined
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
KPI Partners, A global consulting firm focused on strategy, technology, and digital transformation. We help companies tackle their most ambitious projects and build new capabilities. We provide solutions in Cloud, Data, Application Development & BI spaces.
We enable your growth
At KPI, you can become who you want to be and learn skills that will take you further in your career
Continuously upgrade yourself
Develop as a future leader
Drive cloud enablement around the world
Engineering Excellence
Enhance your engineering expertise with our unique approach
This program gives engineers the opportunity to excel in product and software engineering by learning our industry-leading practices, tools, and technologies to build excellence by enhancing their competencies and skills
Visit to Know more :
Title: Sr/Lead Data Engineer 12 Months Contract
Location: 100% Remote PST Time zone (Only W2)
Must Have: PySpark, Strong in SQL, Python / PySpark, ETL and data warehousing fundamentals.
Must have Skills :
8+ Years of total experience collaborating with cross-functional teams, including machine learning engineers, software developers, analytics engineers, and product managers to translate business requirements into highly available data solutions
3+ years of experience developing and implementing data pipelines using PySpark.
Proven experience with SQL
Be an independent self-learner with the let s get this done approach and ability to work in a fast-paced and Dynamic environment
Good-to-Have Skills:
Good to have Databricks Certification
Good to have Experience providing practical direction within Azure native services, implementing data migration and data processing using Azure services: ADLS, Azure Data Factory, Synapse/DW /Azure SQL DB, Fabric.
Good to have Event Hub, IoT Hub, Azure Stream Analytics, Azure Analysis Service, and Cosmo Db knowledge.
Good to have SAP Hana knowledge
Intermediate knowledge of Power BI
Good to have knowledge in DevOps and CI/CD deployments, Cloud migration methodologies and processes