Data Engineer with Medicaid Data Warehouse (AWS)
Posted 5 days ago by 1752660379
Negotiable
Outside
Remote
USA
Summary: We are seeking a highly experienced Data Engineer to develop and maintain an AWS-based Data Warehouse for advanced business analytics in a Medicaid data environment. The role involves managing ETL/data pipeline jobs, transforming data for DB2 Warehouse schemas, and ensuring data quality for Cognos reports. The engineer will utilize IBM DataStage, DB2 Warehouse, and AWS-native services to deliver reliable data solutions. This position is remote and classified as outside IR35.
Key Responsibilities:
- Design, develop, and maintain ETL pipelines using IBM DataStage to ingest and transform Medicaid-related data.
- Implement batch and incremental data jobs to populate DB2 Warehouse target tables in alignment with reporting and analytics requirements.
- Support data modeling activities to ensure data structures meet the needs of Cognos reports and business users.
- Collaborate with business analysts, report developers, and data stewards to translate reporting requirements into data solutions.
- Conduct data profiling, validation, and cleansing activities to ensure data accuracy, completeness, and consistency.
- Utilize AWS services (e.g., S3, EC2, Glue, Lambda) where applicable to enhance data processing and orchestration capabilities.
- Monitor data job performance and troubleshoot issues to maintain operational SLAs.
- Maintain technical documentation and data flow diagrams for ETL jobs and target schemas.
- Support testing, deployment, and version control of data jobs in DEV, TEST, and PROD environments.
- Ensure compliance with HIPAA and Medicaid data security and privacy requirements.
Key Skills:
- Minimum 7 years of experience in Data Engineering, ETL development, or Data Warehousing.
- Hands-on experience with IBM DataStage or a similar ETL tool in a production environment.
- Proficiency with DB2 Warehouse including writing complex SQL queries, stored procedures, and schema design.
- Familiarity with AWS Cloud architecture and services relevant to data processing (e.g., S3, Glue, Lambda, CloudWatch).
- Strong analytical and troubleshooting skills; ability to work with large datasets and optimize performance.
- Experience supporting Cognos or similar BI/reporting tools in data preparation.
- Understanding of data governance, metadata management, and regulatory compliance (e.g., HIPAA).
- Proficiency in shell scripting or Python for job orchestration is a plus.
Salary (Rate): undetermined
City: undetermined
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Job Summary:
We are seeking a highly experienced Data Engineer to support the development and maintenance of an AWS-based Data Warehouse that enables advanced business analytics and reporting in a Medicaid data environment. The engineer will be responsible for developing and managing ETL/data pipeline jobs, transforming data to align with defined DB2 Warehouse schemas, and ensuring consistent data quality and availability for downstream Cognos reports. The role involves working with IBM DataStage, DB2 Warehouse, and AWS-native services to deliver performant and reliable data solutions.
Key Responsibilities:
- Design, develop, and maintain ETL pipelines using IBM DataStage to ingest and transform Medicaid-related data.
- Implement batch and incremental data jobs to populate DB2 Warehouse target tables in alignment with reporting and analytics requirements.
- Support data modeling activities to ensure data structures meet the needs of Cognos reports and business users.
- Collaborate with business analysts, report developers, and data stewards to translate reporting requirements into data solutions.
- Conduct data profiling, validation, and cleansing activities to ensure data accuracy, completeness, and consistency.
- Utilize AWS services (e.g., S3, EC2, Glue, Lambda) where applicable to enhance data processing and orchestration capabilities.
- Monitor data job performance and troubleshoot issues to maintain operational SLAs.
- Maintain technical documentation and data flow diagrams for ETL jobs and target schemas.
- Support testing, deployment, and version control of data jobs in DEV, TEST, and PROD environments.
- Ensure compliance with HIPAA and Medicaid data security and privacy requirements.
Required Qualifications:
- Minimum 7 years of experience in Data Engineering, ETL development, or Data Warehousing.
- Hands-on experience with IBM DataStage or a similar ETL tool in a production environment.
- Proficiency with DB2 Warehouse including writing complex SQL queries, stored procedures, and schema design.
- Familiarity with AWS Cloud architecture and services relevant to data processing (e.g., S3, Glue, Lambda, CloudWatch).
- Strong analytical and troubleshooting skills; ability to work with large datasets and optimize performance.
- Experience supporting Cognos or similar BI/reporting tools in data preparation.
- Understanding of data governance, metadata management, and regulatory compliance (e.g., HIPAA).
- Proficiency in shell scripting or Python for job orchestration is a plus.
Preferred Qualifications:
- Experience with Medicaid data and Medicaid Data Warehouses is strongly preferred.
- AWS certification (e.g., AWS Certified Data Analytics Specialty, AWS Solutions Architect Associate).
- IBM Data Stage Certifications
- Familiarity with Agile/Scrum methodologies and Jira/Confluence tools.