AI Agent Developer Microsoft Copilot Studio

AI Agent Developer Microsoft Copilot Studio

Posted Today by 1752383907

Apply
Negotiable
Outside
Remote
USA
Apply
Artificial Intelligence Automation Azure DevOps Azure Monitor Budgeting Continuous Integration and Continuous Delivery DevOps General Insurance Github Insurance Policies Kusto Query Language Microsoft 365 Microsoft Azure Role-Based Access Control (RBAC) Software Development Stakeholder Management Version Control

Summary: The role of AI Agent Developer for Microsoft Copilot Studio involves designing and operationalizing AI agents using the Copilot Studio platform. The candidate will lead the full lifecycle of agent development, ensuring governance, observability, and cost optimization while collaborating with various teams. This position requires extensive experience in Generative AI and hands-on expertise with Microsoft Copilot Studio. The developer will also mentor others and establish best practices within the organization.

Key Responsibilities:

  • Stand up Copilot Studio and Microsoft AI Foundry environments, establishing reference architectures and configuration baselines.
  • Design, prompt-engineer, and build AI Agents to process large document corpora and implement pipelines between Copilot Studio and AI Foundry.
  • Create promotion pipelines with version control, automated testing, and rollback strategies, defining branching and release strategies.
  • Instrument copilots with logging, telemetry, and usage analytics, building dashboards to track agent performance.
  • Enforce data-access controls and optimize costs while ensuring quality in Azure consumption.
  • Mentor developers and run working sessions to share best practices and document lessons learned.

Key Skills:

  • Proven production experience building Copilot Studio agents end-to-end.
  • Strong grasp of AI Foundry, vector stores, and grounding strategies.
  • Expertise in prompt engineering and evaluation techniques.
  • Fluency with Azure DevOps / GitHub Actions for CI/CD and Infrastructure-as-Code.
  • Implemented observability stacks and demonstrated cost-management of GenAI workloads.
  • Excellent communication skills for translating technical concepts to business stakeholders.

Salary (Rate): undetermined

City: undetermined

Country: USA

Working Arrangements: remote

IR35 Status: outside IR35

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:
  • Experience Level:8+ years overall, 3+ years in Generative AI / LLM solutions, 1+ year hands-on with Microsoft Copilot Studio building and operationalizing agents in production

Position Overview

We are looking for a hands-on architect-developer who can stand up Microsoft Copilot Studio platform, design repeatable patterns for AI Agent development, and lead the full lifecycle from ideation through production deployment. You will be the first dedicated Copilot Studio specialist on the team, partnering with other teams to build governed, observable, and cost-optimized agents that deliver immediate business value and scaling to an enterprise platform.

Key Responsibilities

Area

What You ll Do

Architecture & Platform Setup

Stand up Copilot Studio and Microsoft AI Foundry environments (networking, security, RBAC, data connections, CI/CD).
Establish reference architectures, naming standards, and configuration baselines for new copilots.

Agent Development

Design, prompt-engineer, and build AI Agents in Copilot Studio to ingest, reason over, and act on large document corpora (SharePoint, email, insurance policies, safety plans).
Implement seamless push / pull pipelines between Copilot Studio and AI Foundry.

Dev Test Prod Automation

Create repeatable promotion pipelines with version control, automated testing, and rollback strategies.
Define branching and release strategies that fit SDLC.

Observability & Monitoring

Instrument copilots with robust logging, telemetry, and usage analytics (quality, latency, cost per request).
Build dashboards / alerts to track agent health, hallucination rates, accuracy, response times, and user satisfaction.

Governance & Cost Optimization

Enforce data-access controls, privacy policies, and prompt-safety gates.
Tune prompts, embeddings, and capacity planning to minimize Azure consumption without compromising quality.

Knowledge Transfer & Leadership

Mentor developers and automation engineers; run working sessions, share best practices, and codify lessons-learned into internal playbooks.

Must-Have Qualifications

  • Proven production experience building Copilot Studio agents end-to-end (prompt design deployment monitoring).
  • Strong grasp of AI Foundry, vector stores, Retrieval-Augmented Generation, and grounding strategies.
  • Expertise in prompt engineering, evaluation techniques (BLEU, semantic similarity, human-in-the-loop).
  • Fluency with Azure DevOps / GitHub Actions for CI/CD, Infrastructure-as-Code (Bicep/Terraform/ARM, or similar).
  • Implemented observability stacks (App Insights, Azure Monitor, OpenTelemetry, Log Analytics, KQL, or similar).
  • Demonstrated cost-management of GenAI workloads (token budgeting, capacity reservations, caching).
  • Excellent communication skills; able to translate technical concepts for business stakeholders and teach peers.

Nice-to-Have

  • Experience integrating Microsoft Purview for policy enforcement and data-lineage of LLM pipelines.
  • Familiarity with SharePoint, Exchange / Graph API, and insurance domain terminology (policy, loss control).
  • Hands-on with LangChain, Semantic Kernel, or similar frameworks but comfortable building natively in Azure.
  • Experience with multi-models including GPT4, GPT 4o, GPT 3.5, o3, Llama, and other models.

Pardha S S Moyida

Apply
Similar Jobs
Loading data...