Azure Monitor Agent on Batch pool VMSS returns InvalidResourceId (401) — VMSS resource ID does not match expected Batch pool resource ID

Question

Summary

When installing Azure Monitor Agent (AMA) on Azure Batch pool nodes via Azure Policy, the agent fails to authenticate with a 401 InvalidResourceId error. The resource ID presented in the access token (the underlying VMSS resource ID) does not match the resource ID expected by the Azure Monitor control plane (the Batch pool resource ID).

Goal

I'm trying to collect OS-level metrics (CPU, memory, disk I/O) from Azure Batch pool nodes and ship them to a Log Analytics workspace using Azure Monitor Agent, without modifying application code. Since I'd like to capture metrics for hundreds of nodes, I'm looking for a scalable solution that does not require manual configuration on each node, hence why I wanted to test a proof-of-concept using Azure Policies and AMAs.

Environment

Azure Batch pool using Virtual Machine Configuration (not Cloud Services)
Pool nodes provisioned as a VM Scale Set (Microsoft.Compute/virtualMachineScaleSets) in an auto-generated resource group separate from the Batch account
Node OS: Ubuntu 24.04
Pool type: spot instances
AMA version installed: 1.40.0

Setup

For completeness I'm listing here all the resources I created (all of them in the same Azure subscription and region).

Infrastructure

A test Batch account with a single pool of 2 spot VMs running Ubuntu 24.04
- Note: The Batch account places pool VMs in an auto-generated resource group separate from the Batch account resource group — this is a key detail for policy scoping
A test Log Analytics workspace
A user-assigned managed identity having the following roles:
- Monitoring Metrics Publisher role on the Log Analytics workspace
- Virtual Machine Contributor role on the whole subscription (for simplicity, so I don't have to guess the auto-generated RG name and scope the role assignment at that level)
A Data Collection Rule (DCR) configured to collect Performance Counters, associated to the pool VMSS

Azure Policies

I created 2 new policies assignments (starting from the two built-in policies below) and scoped them to the auto-generated resource group containing the pool VMSS:

Configure Linux virtual machine scale sets to run Azure Monitor Agent with user-assigned managed identity-based authentication
Configure Linux virtual machine scale sets to be associated with a Data Collection Rule

After creating a task remediation both policies show 100% compliant status.

Verified State

AMA extension (AzureMonitorLinuxAgent 1.40.0) installed on the VMSS — confirmed via:

  az vmss extension list --output table
  # AzureMonitorLinuxAgent  Microsoft.Azure.Monitor  1.40.0  ProvisioningState/succeeded

Both VMSS instances have latestModelApplied: True — confirmed via:

  az vmss list-instances --query "[].{instanceId:instanceId, latestModel:latestModelApplied}"
  # 0  True
  # 1  True

AMA service running on both instances — confirmed via:

  az vmss run-command invoke --scripts "sudo systemctl status azuremonitoragent"
  # Active: active (running)
  # Enable succeeded

Problem

Despite all steps completing successfully, no data appears in the Log Analytics workspace — neither Heartbeat nor Perf tables contain any records.

Inspecting the AMA error log on the instances via az vmss run-command invoke reveals the following error repeating every 60 seconds:

Response Code: 401
{
  "error": {
    "code": "InvalidResourceId",
    "message": "Resource ID in the access token
      ('/subscriptions//resourceGroups//providers/Microsoft.Compute/virtualMachineScaleSets/')
      does not match the expected one
      ('/subscriptions//resourceGroups//providers/Microsoft.Batch/batchAccounts//pools/')"
  }
}

Question

Is there an officially supported way to install and run Azure Monitor Agent on Azure Batch pool nodes to collect OS-level VM metrics?

Specifically:

Is there a supported configuration that resolves the InvalidResourceId mismatch between the VMSS resource ID and the Batch pool resource ID?
Is there a recommended alternative approach to collect OS-level metrics (CPU, memory, disk) from Batch pool nodes into a Log Analytics workspace?

Accepted Answer

Francesco Cipolla - Azure Batch pool nodes are logically owned by the Batch pool resource (Microsoft.Batch/batchAccounts/pools)

But physically implemented as a VMSS (Microsoft.Compute/virtualMachineScaleSets)

AMA authenticates using the VM/VMSS resource ID, while

Azure Monitor expects the Batch pool resource ID

This resource identity mismatch cannot be fixed via policy, RBAC, or DCR configuration - https://techcommunity.microsoft.com/blog/azurepaasblog/integrating-azure-monitor-in-azure-batch-to-monitor-batch-pool-nodes-performance/4428929

There is no supported way to “remap” the VMSS identity to the Batch pool identity for AMA.

Option 1: Azure Batch diagnostics - https://learn.microsoft.com/en-us/azure/batch/monitor-batch

Use Batch platform metrics and logs via Diagnostic Settings:
Metrics: node counts, pool state, task execution
Logs: pool resize, task lifecycle events
Destination: Log Analytics

Option 2 (Supported for OS metrics): Install AMA at pool creation time

If you must collect OS-level metrics: - https://techcommunity.microsoft.com/blog/azurepaasblog/integrating-azure-monitor-in-azure-batch-to-monitor-batch-pool-nodes-performance/4428929

Batch account must use User Subscription mode
AMA must be defined at pool creation (ARM template / REST only)
Extensions cannot be added later
Azure Policy remediation is not supported
identity + DCR must be wired during pool creation

Answer

The provided information does not describe any supported configuration for running Azure Monitor Agent directly on Azure Batch pool VMSS nodes, nor any mechanism to change the resource ID that AMA uses so that it matches the Batch pool resource ID. The context only covers:

General AMA behavior and troubleshooting on standard Azure VMs/VMSS (including managed identity and DCR association).
Use of Azure Policy to deploy AMA and associate DCRs to regular VMs/VMSS or Arc machines.
Known issues around managed identities and Azure Policy, but not the InvalidResourceId scenario for Batch pools.

From the context, only the following can be stated:

For standard Azure VMs/VMSS, the supported pattern is:
- Enable a system-assigned or user-assigned managed identity on the VM/VMSS.
- Ensure the AMA extension is installed and in Provisioning succeeded state.
- Associate the VM/VMSS with a DCR in the same region as the Log Analytics workspace.
- Verify that the agent can reach Azure Monitor control plane and IMDS, and that the DCR is downloaded and applied.
These steps are documented for regular VMs/VMSS, not for Batch pool nodes.
Azure Policy is supported to deploy AMA and associate DCRs to existing machines, with some known limitations around managed identities and policy assignment scopes. Again, this is documented for standard Azure resources, not specifically for Batch pool VMSS created and managed by the Batch service.

The context does not contain any statement that:

AMA is officially supported on Azure Batch pool nodes.
The InvalidResourceId mismatch between the underlying VMSS and the Batch pool resource can be configured or overridden.
There is a Batch-specific pattern or alternative (e.g., a dedicated Batch integration with AMA) for collecting OS-level metrics into Log Analytics.

Because of that, the questions about an officially supported way to run AMA on Batch pool nodes, how to resolve the InvalidResourceId mismatch, and any recommended alternative approach for Batch nodes cannot be answered from the provided material.

References:

Share via