Aws Data Architect

What We Are Looking For:

We are seeking a highly skilled and experienced AWS Data Architect who will be responsible for developing and managing data pipelines on AWS.

The ideal candidate will bring strong expertise in AWS data services, big data processing, and data modeling to help us provide high-performance data solutions.

Data Pipeline Development and Management:

• Design, build, and maintain scalable and reliable data pipelines using AWS Glue, Kinesis, and Lambda.
• Develop ETL/ELT processes to process and transform large volumes of structured and unstructured data.
• Optimize data pipeline performance, scalability, and reliability.
• Ensure data processing and ingestion workflows are monitored and meet performance SLAs.

Data Storage and Management:

• Design and implement data storage solutions using Amazon S3, Redshift, DynamoDB, and RDS.
• Optimize data structures, partitioning, and indexing for performance and cost efficiency.
• Ensure data security, integrity, and availability across different AWS storage solutions.
• Manage data lifecycle policies and archiving processes.

Data Transformation and Processing:

• Develop data transformation processes using AWS Glue, EMR, and Athena.
• Implement data quality checks, validation rules, and monitoring solutions.
• Support real-time and batch data processing needs.

Data Integration and Automation:

• Integrate data from multiple sources, including APIs, databases, and third-party applications.
• Automate data ingestion, transformation, and export using tools like Lambda and Step Functions.
• Ensure data consistency across different environments and systems.

Collaboration and Stakeholder Engagement:

• Work closely with data scientists and analysts to understand data needs and business goals.
• Provide technical guidance and best practices to the data engineering and business teams.
• Collaborate with security and compliance teams to ensure data governance standards are met.

Performance Monitoring and Troubleshooting:

• Monitor data pipeline performance and troubleshoot issues in real-time.
• Analyze data pipeline failures and implement fixes to prevent recurrence.
• Set up logging and monitoring using CloudWatch and X-Ray.