SW Engineer ( AI/ML)
Posted 6 days ago by 1756365637
Negotiable
Outside
Remote
USA
Summary: The role of Software Engineer (AI/ML) focuses on leveraging extensive experience in Python/PySpark and the Hadoop ecosystem to optimize data processing jobs. Candidates are expected to design and develop cloud applications while working with large datasets and implementing distributed computing concepts. Strong analytical skills and effective communication are essential for success in this remote position, which operates outside of IR35 regulations.
Key Responsibilities:
- Optimize Python/PySpark jobs in a Hadoop ecosystem.
- Work with large data sets and pipelines using Hadoop tools and libraries.
- Design and develop cloud applications on platforms like AWS or OCI.
- Implement distributed/cluster computing concepts.
- Develop multi-threaded applications focusing on concurrency and performance tuning.
- Create and consume RESTful Web Services.
- Utilize relational and NoSQL databases effectively.
- Apply knowledge of CI tools and microservices architecture.
- Participate in Agile/Scrum methodologies and project management using tools like JIRA.
- Demonstrate strong analytical and communication skills.
Key Skills:
- 5+ years in Python/PySpark.
- 5+ years optimizing jobs in a Hadoop ecosystem.
- 5+ years with large data sets and Hadoop tools (Spark, HDFS, YARN, Hive, Oozie).
- 5+ years designing cloud applications (AWS, OCI).
- 5+ years in distributed computing concepts.
- 5+ years with relational databases (MS SQL Server or similar).
- 3+ years with NoSQL databases (HBASE preferred).
- 3+ years in RESTful Web Services.
- 5+ years in multi-threaded applications and performance tuning.
- Strong knowledge of shell scripting and file systems.
- Familiarity with CI tools (Git, Maven, Jenkins).
- Knowledge of microservices and container orchestration (Docker, Kubernetes).
- Experience with Agile/Scrum methodologies.
- Strong analytical and communication skills.
Salary (Rate): undetermined
City: undetermined
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Job Requirements -
Qualifications:
Software Engineer Big Data
5+ years in Python/PySpark
5+ years optimizing Python/PySpark jobs in a hadoop ecosystem
5+ years working with large data sets and pipelines using tools and libraries of Hadoop ecosystem such as Spark, HDFS, YARN, Hive and Oozie.
5+ years with designing and developing cloud applications: AWS, OCI or similar.
5+ years in distributed/cluster computing concepts.
5+ years with relational databases: MS SQL Server or similar
3+ years with NoSQL databases: HBASE (preferred)
3+ years in creating and consuming RESTful Web Services
5+ years in developing multi-threaded applications; Concurrency, Parallelism, Locking Strategies and Merging Datasets.
5+ years in Memory Management, Garbage Collection & Performance Tuning.
Strong knowledge of shell scripting and file systems.
Preferred: Knowledge of CI tools like Git, Maven, SBT, Jenkins, and Artifactory/Nexus
Knowledge of building microservices and thorough understanding of service-oriented architecture
Knowledge in container orchestration platforms and related technologies such as Docker, Kubernetes, OpenShift.
Understanding of prevalent Software Development Lifecycle Methodologies with specific exposure or participation in Agile/Scrum techniques
Strong knowledge and application of SAFe agile practices, preferred.
Flexible work schedule.
Experience with project management tools like JIRA.
Strong analytical skills
Excellent verbal, listening and written communication skills
Ability to multitask and prioritize projects to meet scheduled dea