Azure Data Engineer

Key Responsibilities

• Design, develop, and optimize data pipelines using Azure Data Factory, Databricks, PySpark, and SQL.
• Build and manage data solutions using ADLS, Snowflake, and Azure services.
• Implement real-time data processing using Kafka, CDC, and event-driven architecture.
• Support GenAI initiatives involving RAG, prompt engineering, and LLM fine-tuning.
• Automate deployments and infrastructure using Terraform, GitHub Actions, and CI/CD.
• Manage workflow orchestration, security, and cloud integrations.

Required Skills

• Strong Azure Data Engineering experience.
• Databricks, Spark/PySpark, Python/Scala, SQL, and Snowflake.
• Azure Data Factory (ADF), ADLS, Azure Functions, and Key Vault.
• Kafka, streaming, CDC, and event-driven architecture.
• Airflow, GitHub, GitHub Actions, Terraform, and CI/CD.
• Exposure to RAG, prompt engineering, and LLM fine-tuning.