AI Agent Developer

Key responsibilities

• Design and implement agents and multi-agent workflows using the Microsoft Agent Framework and Azure AI Foundry, including single-agent, supervisor/worker, sequential and handoff orchestration patterns.
• Build the agent orchestration layer: intent routing, planning, tool selection, conversation and thread state, human-in-the-loop checkpoints, retries, timeouts and graceful degradation.
• Define and implement tool/function calling contracts against the Agent Data API and Member Data API, including strict schemas, input validation, authorisation checks and safe handling of partial or missing data.
• Design and tune the retrieval-augmented generation pipeline over Azure AI Search - chunking, embeddings, hybrid and semantic search, filters, reranking, query rewriting and citation of sources - across the SharePoint, Azure Data Lake and GitHub knowledge base.
• Engineer, version and test system prompts, instructions and grounding strategies; select and configure the appropriate Azure AI models for each task with an eye on quality, latency and cost.
• Implement persistent agent state, memory and audit trails in Azure Cosmos DB via the Agent Data API.
• Build the evaluation harness: golden datasets, groundedness, relevance, task-completion and safety metrics, offline and online evaluation, and regression gating of prompt and model changes in CI.
• Integrate guardrails end to end - Azure AI Content Safety, prompt-shield/jailbreak defences, PII handling, refusal and escalation behaviour, and grounding constraints to prevent hallucinated advice.

Essential skills and experience

• Strong software engineering skills in Python and/or C#/.NET, with production code in source control and test coverage.
• Hands-on experience building LLM applications - agents, tool/function calling, RAG or orchestration frameworks (Microsoft Agent Framework, Semantic Kernel, AutoGen, LangChain, LangGraph or equivalent).
• Practical prompt engineering skill, and the judgement to know when a prompt change is the wrong fix.
• Experience with Azure OpenAI/Azure AI Foundry or a comparable managed LLM platform, including model selection and parameter tuning.
• Retrieval and vector search experience: embeddings, chunking strategies, hybrid search, relevance tuning (Azure AI Search preferred).
• Experience consuming and designing against REST APIs, with secure token-based authentication.
• A rigorous approach to evaluating non-deterministic systems - you measure quality rather than asserting it.
• Awareness of LLM risk: hallucination, prompt injection, data leakage, over-permissioned tools, and how to mitigate each.
• Git-based collaborative development and CI/CD familiarity.
• Production experience with the Microsoft Agent Framework or Azure AI Foundry Agent Service specifically.
• Experience with Azure AI Content Safety, prompt shields or another guardrail stack.
• Azure Cosmos DB data modelling.
• Certifications such as AI-102 or equivalent.