Job Overview
We are seeking a highly skilled Senior Databricks Architect/Engineer to lead the design, development, and implementation of a next-generation data platform. This role will be instrumental in building a scalable Databricks-based solution to support treasury functions such as cash management, liquidity, forecasting, and payments, while enabling future BI and AI capabilities.
Key Responsibilities
- Design and architect a scalable Databricks-based data platform from the ground up
- Perform hands-on development using Databricks, PySpark, and SQL
- Build and optimize data pipelines to ingest data from Azure/SQL Server environments
- Implement data architecture frameworks (e.g., medallion architecture)
- Configure and optimize Databricks clusters for performance and cost efficiency
- Collaborate with stakeholders to gather requirements and translate them into technical solutions
- Provide technical leadership and guidance to development teams
- Ensure platform stability through monitoring, optimization, and best practices
- Support integration with downstream tools for BI and AI use cases
Required Skills & Experience
- Strong hands-on experience with Databricks (Azure preferred)
- Proficiency in PySpark, SQL, and data engineering concepts
- Experience designing and implementing data platforms and architectures
- Solid understanding of Azure services (ADF, ADLS, SQL Server, etc.)
- Experience with data pipeline frameworks and ETL/ELT processes
- Knowledge of cluster management, performance tuning, and optimization
- Strong communication and stakeholder management skills
- Ability to work independently in a remote environment
- Experience in financial or treasury data domains (cash, liquidity, payments)
- Familiarity with Delta Lake, Unity Catalog, and data governance practices
- Exposure to BI tools and AI/ML integration
Client Feedback on the Recent Candidate (John)
John is a no. Not hands on enough. Here is the interview feedback. Need someone with Architecture and solid engineering/hands on/building experience.
Just wrapped up with John. He is good but I found him to be high level. He is aware of our technology, understands our requirement and could also create a high level architecture but wouldn't be able to build and implement it. He also lacked data modeling and data architecture knowledge, which are pretty basic for lakehouse/BI implementation. I am passing him.
Recruiting Notes
- Seeking candidates with strong hands-on architecture and engineering experience.
- Must have proven experience building and implementing solutions, not just designing high-level architectures.
- Strong data modeling and data architecture skills are required.
- Experience with Lakehouse and BI implementations is highly preferred.
- Looking for candidates with a balance of architectural expertise and hands-on technical execution.
Notes
- Architecture and hands on development
- Databricks is new at ripple treasury
- Will be their data platform of the future
- built on Azure/SQL server
- for cash, liquidity, forecasting, payments
- bring the data into Databricks and use platform for BI and AI in the future
- need someone who has data background and knowledgeable to help with architecture, design, and developers
- know frameworks used in Databricks
- help with clustering
- person would also lead the buildout of the platform (from a functional subject matter expert)
- will build and then stabilize the platform