Senior Data Engineer

Job Responsibilities

• Build and operate production-grade batch and streaming data pipelines across SQL Server, cloud applications, device/event streams, Customer Relationship Management systems, and third-party sources.
• Develop ingestion, transformation, and enrichment pipelines using Apache Flink, Kafka, Change Data Capture, SQL, and Python/Java/Scala.
• Implement lakehouse data layers across landing, raw, conformed, and consumption zones using Apache Iceberg and Trino.
• Build dimensional and analytical models for entities such as buildings, tenants, credentials, devices, and people.
• Implement robust handling for Change Data Capture, late-arriving data, retries, backfills, replay, schema evolution, and data reconciliation.
• Build and maintain integration between the lakehouse and Snowflake, including Iceberg-backed/external tables, secure sharing, and downstream data contracts.
• Develop automated data-quality checks, pipeline monitoring, lineage, freshness alerts, and operational dashboards.
• Troubleshoot and resolve production data issues, pipeline failures, performance bottlenecks, and data-quality problems.
• Optimize SQL queries, storage layouts, partitioning, clustering, compute utilization, and overall platform cost.
• Implement security controls including role-based access, row/column-level security, masking, and protection of sensitive data.
• Build and maintain Continuous Integration and Continuous Deployment pipelines, automated testing, infrastructure-as-code, and environment promotion processes.
• Participate in production support, incident resolution, Root Cause Analysis, deployment, and on-call activities.
• Work closely with Product, Application Engineering, Quality Assurance, DevOps/Site Reliability Engineering, Analytics, and globally distributed engineering teams.
• Contribute to technical design decisions and documentation while remaining primarily hands-on in implementation and production delivery.

Required Skills

• 6+ years of hands-on experience in data engineering, data warehousing, or data platform development.
• Strong production experience building and supporting end-to-end data pipelines.
• Expert SQL, including complex queries, query-plan analysis, and performance tuning.
• Strong development skills in Python, Java, or Scala.
• Hands-on experience with Apache Iceberg, Delta Lake, or Apache Hudi.
• Experience with distributed query technologies such as Trino, Presto, or Spark SQL.
• Hands-on Snowflake experience, including data modeling, performance optimization, external/Iceberg tables, secure data sharing, or access management.
• Experience with streaming and Change Data Capture technologies such as Apache Flink, Kafka, Spark Structured Streaming, or Debezium.
• Strong understanding of dimensional modeling, data quality, schema evolution, and data contracts.
• Experience with Kubernetes, containers, Continuous Integration and Continuous Deployment, and infrastructure-as-code.
• Experience implementing monitoring, alerting, lineage, reconciliation, and operational support for production data platforms.
• Ability to independently troubleshoot complex production issues and drive them through resolution.

Preferred Qualifications

• Experience with multi-tenant Software as a Service data platforms and customer data isolation.
• Experience integrating open lakehouse platforms with Snowflake.
• Azure cloud experience.
• Experience with Airflow, Dagster, dbt, Power BI, Looker, or Superset.
• Experience with data governance, cataloging, masking, and access-control frameworks.
• Exposure to Internet of Things/device telemetry, physical security, commercial real estate, or property technology.
• Experience using Artificial Intelligence-assisted engineering tools to accelerate data development.