Senior AI Data Engineer

Senior AI Data Engineer

Posted Today by 1759916371

Negotiable
Outside
Hybrid
USA

Summary: The role of Senior AI Data Engineer involves working with Azure and Databricks to build and optimize data workflows and pipelines. The position requires strong experience in cloud computing and data engineering, particularly with ETL/ELT processes. Candidates with Gen AI experience will have an advantage. This is a long-term contract opportunity based in Baltimore, MD, with remote work options available.

Key Responsibilities:

  • Build and optimize workflows and pipelines using Databricks and Spark (PySpark preferred).
  • Design, build, and maintain ETL/ELT pipelines using Python, PySpark, and related tools.
  • Utilize strong SQL skills for data manipulation and querying.
  • Work with cloud services (AWS, Azure, or Google Cloud Platform) and cloud-native data tools.
  • Handle big data systems and integrate data from diverse sources (e.g., APIs, flat files, databases).
  • Configure secure external connections to data sources using service principals, SAS tokens, OAuth, or mounting techniques.
  • Ensure data security, encryption, and compliance standards are met.
  • Implement data governance frameworks to manage data quality.
  • Use Azure Data Factory (ADF) for orchestrating and automating data workflows across cloud and on-premises systems.

Key Skills:

  • Hands-on advanced experience with Databricks and Spark (PySpark preferred).
  • Proficiency in designing, building, and maintaining ETL/ELT pipelines using Python and PySpark.
  • Strong SQL skills.
  • Experience with cloud services (AWS, Azure, or Google Cloud Platform).
  • Expertise in handling big data systems and integrating data from diverse sources.
  • Experience in configuring secure external connections to data sources.
  • Strong understanding of data security, encryption, and compliance standards.
  • Experience implementing data governance frameworks.
  • Proven expertise in using ADF for orchestrating data workflows.
  • Knowledge of Data Vault methodology and familiarity with workflow orchestration tools like FiveTran (nice-to-have).

Salary (Rate): undetermined

City: Baltimore

Country: USA

Working Arrangements: hybrid

IR35 Status: outside IR35

Seniority Level: Senior

Industry: IT

Detailed Description From Employer:

RESPONSIBILITIES:
Kforce's client in Baltimore, MD is seeking an experienced Azure Data Engineer with solid Databricks experience to join their team on a long term contract. Gen AI experience/projects are a plus and will come into play as well. Sound/Strong experience in cloud computing in general is needed.

REQUIREMENTS:
* Hands-on advanced experience with Databricks and Spark (PySpark preferred) for building and optimizing workflows and pipelines
* Hands-on proficiency in designing, building, and maintaining ETL/ELT pipelines using Python, PySpark, and related tools
* Strong SQL skills
* Hands-on experience with cloud services (AWS, Azure, or Google Cloud Platform) and cloud-native data tools
* Expertise in handling big data systems and integrating data from diverse sources (e.g., APIs, flat files, databases)
* Experience in configuring secure external connections to data sources (e.g., Azure Data Lake Storage, On-prem SQL Server) using service principals, SAS tokens, OAuth, or mounting techniques
* Strong understanding of data security, encryption, and compliance standards
* Experience implementing data governance frameworks to manage data quality
* Proven expertise in using ADF for orchestrating and automating data workflows across cloud and on-premises systems

Nice-to-Have Skills:
* Knowledge of Data Vault methodology for designing scalable and flexible data models
* Familiarity with workflow orchestration tools like FiveTran

The pay range is the lowest to highest compensation we reasonably in good faith believe we would pay at posting for this role. We may ultimately pay more or less than this range. Employee pay is based on factors like relevant education, qualifications, certifications, experience, skills, seniority, location, performance, union contract and business needs. This range may be modified in the future.

We offer comprehensive benefits including medical/dental/vision insurance, HSA, FSA, 401(k), and life, disability & ADD insurance to eligible employees. Salaried personnel receive paid time off. Hourly employees are not eligible for paid time off unless required by law. Hourly employees on a Service Contract Act project are eligible for paid sick leave.

Note: Pay is not considered compensation until it is earned, vested and determinable. The amount and availability of any compensation remains in Kforce's sole discretion unless and until paid and may be modified in its discretion consistent with the law.

This job is not eligible for bonuses, incentives or commissions.

Kforce is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.

By clicking ?Apply Today? you agree to receive calls, AI-generated calls, text messages or emails from Kforce and its affiliates, and service providers. Note that if you choose to communicate with Kforce via text messaging the frequency may vary, and message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You will always have the right to cease communicating via text by using key words such as STOP.