Audit & Traceability Engineer / AI

We are looking for a hands-on Audit & Traceability Engineer to design and implement end-to-end auditability, data lineage, observability, and evidence management for AI/agentic workflows supporting Medicaid Record-to-Report (R2R) processes. The role will ensure that data, calculations, agent decisions, tool/API calls, exceptions, and human approvals are traceable, reproducible, explainable, and audit-ready.

Key Responsibilities

• Implement end-to-end data lineage and decision provenance from source systems through agent workflows and human approvals.
• Build structured audit logging, telemetry, tracing, and monitoring for AI agents and APIs.
• Capture agent execution details including prompt/model versions, tool calls, inputs/outputs, errors, retries, and workflow states.
• Enable historical replay and reproducibility through versioned data, rules, prompts, models, and configurations.
• Implement traceability for exceptions, reconciliations, approvals, overrides, and control evidence.
• Develop automated tests for lineage completeness, reconciliation, regression, security controls, and failure recovery.
• Ensure audit traces comply with HIPAA, PHI/PII protection, RBAC, least privilege, and data-retention requirements.
• Collaborate with AI, data, Medicaid, finance, security, QA, and Controller teams.

Required Technical Skills

• Strong Python and SQL
• Data lineage, metadata, and provenance
• Audit logging and distributed tracing
• AI/LLM and agentic workflow traceability
• REST APIs and integration
• Workflow orchestration and state management
• Observability, telemetry, and monitoring
• Automated testing and reconciliation
• IAM/RBAC, encryption, secrets management, and secure logging
• Understanding of LLMs, prompts, tool/function calling, human-in-the-loop, and AI guardrails

Preferred Technology Experience

• Azure
• Databricks / Delta Lake
• Unity Catalog
• MLflow tracing and evaluation
• Azure OpenAI
• LangChain / LangGraph or equivalent agent frameworks
• MCP or similar agent-to-tool integration patterns

Healthcare / Medicaid Knowledge: Working knowledge of:

• Medicaid enrollment and eligibility
• Capitation and reconciliation
• Claims and encounters
• 834, 820, 837, and 835 healthcare transactions
• HIPAA and PHI/PII controls
• Healthcare financial/R2R processes