Machine Learning Infrastructure Engineer
Posted 5 days ago by 1756443692
Negotiable
Outside
Remote
USA
Summary: The role of Machine Learning Infrastructure Engineer involves designing and implementing evaluation frameworks for large language models, developing tools for user feedback, and maintaining ML infrastructure. This position is fully remote and requires collaboration with cross-functional teams to enhance model performance and usability. Candidates should have extensive experience in ML infrastructure and strong programming skills in Python. The role is based in the greater Portland ME area, but it allows for remote work.
Key Responsibilities:
- Implement and design LLM evaluation frameworks to support human-in-the-loop and automated assessment of model performance.
- Build custom feedback tools to collect unstructured and structured user feedback on model predictions.
- Develop systematic analysis tools for logged predictions, enabling deep dives into model behavior, performance trends, and error patterns.
- Maintain and create tooling and infrastructure that supports the end-to-end machine learning lifecycle, including training, evaluation, data preparation, annotation, and monitoring.
- Ensure reliability, scalability, and usability of ML infrastructure across projects and teams. Collaborate with cross-functional teams to integrate feedback and evaluation tools into production ML pipelines.
Key Skills:
- 3+ years of experience in ML infrastructure, MLOps, or backend engineering for ML systems
- Strong programming skills in Python and experience with ML/DS libraries (e.g. TensorFlow, PyTorch, scikit-learn)
- Deep understanding of ML evaluation methodologies, especially for Large language Models and generative models
- Experience with Databricks for model training, data engineering, and collaborative workflows
- Hands-on experience with SuperAnnotate or similar data annotation platforms
- Familiarity with AWS services for scalable ML infrastructure
- Familiarity with monitoring, logging, and observability tools for ML systems
Salary (Rate): undetermined
City: undetermined
Country: USA
Working Arrangements: remote
IR35 Status: outside IR35
Seniority Level: undetermined
Industry: IT
Could you be a good fit? We are looking for the best, highly skilled Machine Learning Infrastructure Engineer to join the team of our exceptional client located in the greater Portland ME area. This is a fully remote role.
Benefits:
- Remote
- Healthcare Medical including HSA
- Dental and Vision Insurance
- 401k
What will your day look like? As a Machine Learning Engineer you will:
- Implement and design LLM evaluation frameworks to support human-in-the-loop and automated assessment of model performance.
- Build custom feedback tools to collect unstructured and structured user feedback on model predictions.
- Develop systematic analysis tools for logged predictions, enabling deep dives into model behavior, performance trends, and error patterns.
- Maintain and create tooling and infrastructure that supports the end-to-end machine learning lifecycle, including training, evaluation, data preparation, annotation, and monitoring.
- Ensure reliability, scalability, and usability of ML infrastructure across projects and teams. Collaborate with cross-functional teams to integrate feedback and evaluation tools into production ML pipelines.
You will be a good fit for the Machine Learning Infrastructure Engineer role if you have:
- 3+ years of experience in ML infrastructure, MLOps, or backend engineering for ML systems
- Strong programming skills in Python and experience with ML/DS libraries (e.g. TensorFlow, PyTorch, scikit-learn)
- Deep understanding of ML evaluation methodologies, especially for Large language Models and generative models
- Experience with Databricks for model training, data engineering, and collaborative workflows
- Hands-on experience with SuperAnnotate or similar data annotation platforms
- Familiarity with AWS services for scalable ML infrastructure
- Familiarity with monitoring, logging, and observability tools for ML systems
About Market Street Talent
We are a specialized staffing and consulting firm focused on staffing solutions for Information Technology positions. We are heavily invested in the IT vertical and that s what our clients and candidates love about us. We understand the nuances of IT placement the platforms, positions, background and the intricacies of specific jobs. We know what it takes to find highly skilled technology specialists that fit seamlessly into a company s culture.
Our Vision: is to promote and foster the growth of information technology (IT) in our world, one candidate, one client, one community at a time. Our Goal: To coach clients and candidates through the entire placement process and cultivate long term healthy business relationships. Our Culture: At MST, we believe in pursuing excellence in everything we do, treating everyone with the utmost respect, and showing empathy for our community.
Sound like you? Ready for your next challenging IT opportunity? Click Apply Now!