Job Description
The Client, is seeking a Senior Data Engineer / Applied Data Scientist to design, develop, and maintain data pipelines supporting semantic search and AI-enabled applications.
Our search environment supports administrative and research applications that integrate information from content management systems, websites, documents, structured databases, datasets, and internal applications. We are expanding our search capabilities to include semantic and hybrid retrieval, organizational master-data enrichment, and integration with AI agents through the Model Context Protocol (MCP).
The successful candidate will bring demonstrated experience developing production pipelines for semantic search, including document processing, embedding generation, vector indexing, and retrieval evaluation. This position combines approximately 60% data engineering and 40% applied data science.
The individual will work with an existing engineering team responsible for search strategy, infrastructure, and traditional keyword-based indexing. The role requires the ability to independently design and implement technical solutions while actively participating in a scrum team and collaborating on overall system architecture, solutioning, integration, and continuous delivery.
Required Qualifications
- 2+ years of experience with document chunking, embedding generation, vector indexing, and semantic retrieval.
- Experience evaluating, benchmarking, and fine-tuning embedding models for domain-specific applications.
- 2+ years of experience with production vector databases or search platforms such as Qdrant, Milvus, Weaviate, Pinecone, Solr, Elasticsearch, or OpenSearch.
- 2+ years of experience with NLP, entity extraction, entity resolution, and integration of structured master data.
- Hands-on experience developing MCP servers and tools, integrating LLMs, and building agent-based workflows.
- Strong Python and SQL programming skills, including data transformation, API integration, and service development.
- Ability to independently design solutions, evaluate alternatives, and collaborate with multidisciplinary engineering teams.
- U.S. citizenship and ability to obtain a security clearance are required.
Qualifications will be evaluated primarily on demonstrated technical accomplishments and relevant production experience rather than specific degrees or years of service.
Nice to have
- Experience with Apache Solr, Elasticsearch, Qdrant, or hybrid keyword/vector retrieval systems.
- Experience developing REST APIs and services.
- Experience implementing document-level access controls and preserving source-system permissions in data pipelines.
- Familiarity with containerized applications, automated testing, CI/CD, monitoring, and production troubleshooting.
- Experience designing multi-agent systems, including tool orchestration, context management, and agent evaluation.
- Experience with Kubernetes, cloud-native data infrastructure, or migration from on-premises systems.
- Familiarity with JavaScript/TypeScript and modern AI application frameworks.
- Experience working with enterprise identity systems, organizational taxonomies, and sensitive or access-controlled information.
- Experience with search relevance evaluation, reranking models, and retrieval-augmented generation (RAG).
Expected Outcomes: During the first year, the successful candidate will contribute to:
- Establishing production semantic-search pipelines for existing research search applications.
- Improving existing ingestion pipelines and integrating authoritative organizational master-data enrichment.
- Developing and deploying MCP tools that make keyword and vector search capabilities available to internal applications and AI agents.
- Implementing agent workflows that support search, discovery, and information retrieval.
- Establishing repeatable methods for evaluating retrieval quality, embedding models, and pipeline reliability.
Work Location and Remote Work Arrangement
- Virtual work arrangements are available for candidates.
- Candidates living within a 100-mile radius of the Lab are preferred.
- Fully remote candidates may be considered, provided they are able to work East Coast hours.
Security Clearance
- U.S. citizenship is required.
- Candidates must be able to obtain a security clearance.
- Interim clearance is sufficient to start.
Interview Process
- First Round: Zoom interview with the Hiring Manager.
- Second Round: Zoom interview with the Technical Team.