Copilot Agent Kit overview

Copilot Agent Kit is a toolkit designed to enhance your experience with Copilot Studio. The Power Customer Advisory Team (Power CAT) created and maintains the kit to equip makers with tools to build, govern, test, and optimize custom agents.

Test capabilities

Copilot Agent Kit allows makers to configure agents, tests, and test sets, and use them to batch test their custom agents. Test runs produce detailed results, including latencies, observed responses, and pass or fail outcomes, to help identify issues and improve quality.

Learn more in Enhance agent testing

Compliance Hub

Compliance Hub helps organizations define and enforce governance policies for Copilot Studio agents at scale. It continuously evaluates agent configurations collected through Agent Inventory against configured controls to identify risks and policy violations.

Administrators can configure thresholds, risk levels, SLA timers, and enforcement actions (manual review, quarantine, delete), while makers are guided to remediate issues through Teams or Outlook notifications.

Learn more in Define and enforce agent compliance with Copilot Agent Kit Compliance Hub

Agent Insights Hub

Agent Insights Hub helps administrators and makers monitor Microsoft Copilot Studio agents with centralized analytics. It aggregates telemetry from Azure Application Insights and conversation transcripts to provide visibility into usage, quality, and operational trends.

Learn more in Agent Insights Hub

Agent Debugger

Agent Debugger helps administrators investigate recorded conversations and understand how agents produced responses. It provides step-by-step diagnostic details to support troubleshooting and optimization.

Learn more in Agent Debugger.

Agent Library

Agent Library helps makers discover, install, and manage prebuilt agent templates and reusable components. It provides a centralized catalog to accelerate development and promote consistency across agent implementations.

Learn more in Agent Library.

Power Shield

Power Shield helps organizations manage connector access in Data Loss Prevention (DLP) policies through an approval-based workflow. Makers can submit connector access requests, and administrators can review, approve, or reject requests based on governance requirements. It also supports compliance questionnaires, notification settings, and scoped requests for Service Trees and Environment Containers.

Learn more in Power Shield.

Rubrics refinement

Rubrics refinement is a powerful capability in Copilot Agent Kit that enables you to create, test, and iteratively improve reusable evaluation standards (rubrics) for AI-generated responses. This feature supports side-by-side testing, scoring consistency, and rubric tuning over time.

Learn more in Rubrics refinement.

Conversation KPIs

Conversation KPIs (key performance indicators) help makers track and analyze custom agent performance. Conversation KPIs complement Copilot Studio's built-in analytics by providing aggregated data in one place.

Learn more in Track agent performance with Conversation KPIs.

SharePoint synchronization

SharePoint synchronization helps makers periodically sync selected content from SharePoint locations to custom agent knowledge bases as files.

Learn more in Improve agent responses with SharePoint synchronization.

Prompt Advisor

Prompt Advisor helps makers create well-structured prompts to improve user input. Makers enter a prompt to get a confidence score, a detailed analysis, and suggested optimized prompts that they can refine and reuse.

Learn more in Refine prompts with Prompt Advisor.

Webchat Playground

Webchat Playground helps makers customize the appearance and behavior of the agent Web Chat, including the colors, fonts, thumbnails, and more.

Learn more in Customize Web Chat appearance with Webchat Playground.

Adaptive Cards Gallery

Adaptive Cards Gallery offers makers Adaptive Card templates for various scenarios. The templates demonstrate the extensibility of Adaptive Card visuals and behavior, and include examples of agent-side interactions.

Learn more in Enhance user engagement with Adaptive Cards Gallery.

Agent Inventory

Agent Inventory provides administrators with a tenant-wide view of all custom agents in their organization. It shows details about the features they're using, authentication mode, knowledge sources, and other key metadata.

Learn more in Monitor agents using Agent Inventory.

Agent Review Tool

The Agent Review Tool is a solution analysis tool that checks your agents for any potential problems or anti-patterns that might negatively affect performance or security. After analysis, the Agent Review Tool provides recommendations to address findings.

Learn more in Analyze agents using the Agent Review Tool.

Conversation Analyzer

Conversation Analyzer helps makers analyze conversation transcripts of their custom agents by using custom prompts to obtain actionable insights.

Learn more in Analyze conversations transcripts.

Agent Value

The Agent Value component is a modular tool for classifying Copilot Studio agents by type, behavior, and value. It helps organizations understand each agent's role, align agents with strategic goals, and evaluate impact.

Learn more in Measure agent value.

Automated testing using pipelines in Power Platform

This feature automates the testing and deployment of Copilot Studio custom agents by using Power Platform Pipelines. The goal is to ensure that agents are automatically validated through testing before promotion across environments.

Learn more in Automate testing and deployment of agents using pipelines in Power Platform.

Next step