Data Engineer

We are seeking a Senior Databricks Engineer to design, build, and optimize scalable data platforms and analytics solutions using Databricks. This role will be responsible for developing robust data pipelines, enabling advanced analytics, and supporting enterprise data modernization initiatives across cloud environments. The ideal candidate will bring deep expertise in Databricks, Apache Spark, Delta Lake, and cloud-native data engineering practices.

Key Responsibilities

• Design, develop, and maintain scalable data ingestion, transformation, and orchestration pipelines using Databricks.
• Build and optimize batch and streaming data solutions leveraging Apache Spark and Delta Lake.
• Implement and support medallion architecture patterns including bronze, silver, and gold layers.
• Collaborate with data architects, analysts, data scientists, and business stakeholders to translate requirements into technical solutions.
• Develop reusable frameworks, notebooks, workflows, and CI/CD processes for Databricks deployments.
• Optimize cluster performance, job execution, and cost efficiency across data workloads.
• Ensure data quality, lineage, governance, and security standards are met.
• Integrate Databricks with cloud storage, data warehouses, APIs, and enterprise systems.
• Troubleshoot production issues and provide performance tuning for large-scale distributed data processing.
• Mentor junior engineers and promote engineering best practices across the team.
• Support platform modernization, migration, and cloud data strategy initiatives.

Preferred Qualifications

• Databricks certification(s), such as Databricks Certified Data Engineer Associate/Professional.
• Experience with Azure Data Factory, Airflow, dbt, Kafka, or similar tools.
• Familiarity with machine learning data pipelines and support for AI/ML workloads.
• Experience working in regulated or enterprise-scale environments.
• Knowledge of data observability and monitoring tools.
• U.S. Citizenship is required
• Active security clearance, or the ability to obtain and maintain a security clearance, is required.

Education and General Experience:

• Bachelors degree with six years of experience.