Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
The Agent details view in Application Insights provides a unified experience for monitoring AI agents across multiple sources, including Azure AI Foundry, Copilot Studio, and third-party agents.
This feature consolidates telemetry and diagnostics, enabling customers to track agent performance, analyze token usage and costs, troubleshoot errors, and optimize your agent's behavior.
Note
Azure Monitor Agent Observability is based on OpenTelemetry Generative AI Semantics.
Prerequisites
- Azure subscription: If you don't have one, create an Azure subscription for free
- Application Insights resource: Create an Application Insights resource to collect and store your agent telemetry.
Get started
Choose your monitoring approach
Getting started looks different depending on how and where you're building your agents.
Managed hosting
Azure AI Foundry: You can collect telemetry from your agentic application using the Azure Monitor OpenTelemetry Distro and the Azure AI Foundry SDK.
Copilot Studio: You can use built-in configuration to emit your telemetry to Azure Monitor, see Connect your Copilot Studio agent to Application Insights.
Self-hosting
Microsoft Agent Framework: If you're building an agent from scratch and are self-hosting, you can use the Microsoft Agent Framework to orchestrate your agent and emit telemetry to Azure Monitor.
Third-party agents: If you built an agent elsewhere, you can emit your telemetry to Azure Monitor using the Azure AI OpenTelemetry Tracer. These agents can also be registered in Azure AI Foundry.
For more information, see:
If you choose to collect full prompt information (for example, using the EnableSensitiveData flag in Agent Framework), you're able to search through prompts in the Search view and read back conversations, including assistant messages, system prompts, and tool usage, in the Transaction Details view.
Tip
- Make sure to give each of your agents a name, so you tell them apart from each other in the Agent details view.
- If your agentic components are part of a larger application, it may make sense to send them to an existing Application Insights resource.
Note
To see your Agents in AI Foundry (in addition to Azure Monitor), you need to connect an Application Insights resource to your Foundry Project.
Set up evaluations
To set up evaluations, there are several approaches.
Batch evaluations:
- Local evaluations with Azure AI Evaluation SDK: Run evaluations on your development machine during testing.
- Cloud evaluations with Azure AI Foundry SDK: Execute evaluations in Azure for larger datasets or team collaboration.
- Azure Foundry Portal-based evaluations: Use the Azure AI Foundry Portal for no-code evaluation workflows.
Continuous evaluations: Set up automated evaluations that run against production traffic to detect quality regressions.
Monitor your AI agents
Access the Agent details view
Once telemetry is flowing to Application Insights:
In the Azure portal, go to your Application Insights resource.
In the navigation menu, select Agents (Preview).
Note
You can also get to the Agent details view from AI Foundry. From your agent, go to the Monitoring tab, then select View in Azure Monitor.
Investigate traces
To drill into specific agent runs:
Select one of the following from the Agent details view:
- View Traces with Agent Runs - See all agent executions
- View Traces with Gen AI Errors - Focus on failed or problematic runs
- Any individual tool call or model in the Tool Calls or Models tiles
The Search overlay displays filtered traces matching your selection.
Use the search capabilities to:
- Sort traces by metrics like Most tokens used to identify expensive operations
- Filter by time range to isolate specific incidents
- Search through prompt content (if sensitive data logging is enabled)
Select any trace to get to the End-to-end transaction details view for comprehensive analysis.
End-to-end transaction details view
The end-to-end transaction details now offer a simple view, which shows agent steps in a clear, story-like fashion, including the invoked agent, underlying LLM, executed tools, and more.
Simple view allows you to quickly find the relevant telemetry and transition to Azure AI Foundry or other tools to make the necessary changes.
Note
To return to the traditional view, select Leave simple view in the top action bar.
In our example, we were researching high token use. Transaction details allow you to identify that large prompt context and/or an expensive model is driving up token use and costs.
Customize monitoring views with Grafana
The Agent details view in Application Insights provides an opinionated, out-of-the-box experience for monitoring your AI agents. For more advanced customization and visualization needs, you can select Explore in Grafana from the top navigation bar on the Agent details view.
Azure Monitor includes pre-built Grafana dashboards specifically designed for Gen AI monitoring to help you get started:
- Agent Framework - Monitor agent execution and performance
- Agent Framework workflow - Track agent workflow patterns and dependencies
- AI Foundry - Visualize AI Foundry-specific metrics and telemetry
These dashboards serve as a starting point for your monitoring strategy. You can customize them by:
- Using different visualization panels to match your preferences
- Editing or creating new queries to surface specific metrics
- Using Save as to create tailored dashboards for your specific environment and use cases
To learn more about using Grafana with Application Insights, see Dashboards with Grafana in Application Insights and Use Azure Monitor Dashboards with Grafana.
Next steps
- Learn how correlation works in Application Insights with Telemetry correlation.
- Explore the end-to-end transaction diagnostic experience that correlates server-side telemetry from across all your Application Insights-monitored components into a single view.