Upgrade infrastructure for Software Defined Networking managed by on-premises tools

Applies to: Hyperconverged deployments of Azure Local running 2311.2 and later; Windows Server 2025, Windows Server 2022

This article provides guidance on safely and securely upgrading infrastructure for Software Defined Networking (SDN) managed by on-premises tools. It also provides troubleshooting guidance to help remediate issues that might occur during the upgrade process.

About upgrading SDN infrastructure

Your SDN deployment consists of several roles and machines, each providing essential services for your environment. To keep your environment secure and up to date, it's required to upgrade the SDN infrastructure, one node at a time.

Before you begin

• Download the ISO image for performing the in-place upgrade.

  • For Azure Local, see Download operating system for Azure Local deployment.

  • For Windows Server, see Install Windows Server from installation media.

• Use Unblock-File and copy the ISO to a file system that your Hyper-V hosts can access, or copy it manually to each Hyper-V host as needed.

• Install the SdnDiagnostics module on the machine where you'll perform the upgrade tasks:

  # Install or update SdnDiagnostics module
  # After installation or update, reload the module to ensure the latest version is active
  
  $moduleName = 'SdnDiagnostics'
  if ($null -eq (get-module -ListAvailable -Name $moduleName)) {
  Install-Module -Name $moduleName -Repository PSGallery -Scope CurrentUser -Force
  } else {
  Update-Module -Name $moduleName -Scope CurrentUser -Force
  }
  
  # Remove any currently loaded instance of the module
  Remove-Module -Name $moduleName -ErrorAction SilentlyContinue
  
  # Import the latest version into the current session
  Import-Module -Name $moduleName -Force
  
  
  # Optional: Confirm the loaded version
  Get-Module -Name $moduleName
  

• After installation, retrieve current SDN fabric environment details and copy the SdnDiagnostics module into the environment:

  $environmentInfo = Get-SdnEnvironmentInfo -NetworkController "<NC_VM>"
  Install-SdnDiagnostics -ComputerName $environmentInfo.FabricNodes
  

• Ensure sufficient space before proceeding. The in-place upgrade requires a minimum of 40 GB of available storage. For VMs, you can increase the VM's VHD size using Windows Admin Center. After resizing the VHD, adjust the partition within the VM using the Resize-Partition or diskpart commands.

Key considerations

• Upgrade components in the following order:

  • Hyper-V hosts
  • Network Controller nodes
  • Load Balancer Multiplexer nodes (optional)
  • Gateway nodes (optional)

• Upgrade the Network Controller to the latest version before you proceed. Older versions might contain known issues that can affect stability during the upgrade process.

• Do not upgrade the gateway until the Network Controller completes cleanup and reboots the gateway.

• Workloads that use Load Balancer Multiplexers (Internal Load Balancers, Load Balancers, Public IPs) or gateways (Layer 3, Generic Routing Encapsulation (GRE), Site-to-Site connections) might experience temporary disruption while services fail over. Schedule the upgrade during a maintenance window and notify users in advance.

Perform in-place upgrade

Use the steps in this section to perform an in-place upgrade of the existing operating system (OS). These steps apply to all SDN nodes.

Mount the media

Mount-DiskImage for Hyper-V hosts

If the ISO file is located on the local file system, you can mount it directly.

1. Locate the ISO file you downloaded earlier.

2. Run the following command to mount the ISO. Make sure to update the drive letter to one that is not already in use.

   Mount-DiskImage -ImagePath "E:\<PATH_NAME>.ISO"
   

Add-VMDvdDrive for VMs

For an in-place upgrade on a VM, use Hyper-V to attach the ISO as a DVD drive to the VM directly. This approach reduces overhead required on the file system of the OS.

1. On the Hyper-V host where the VM resides, locate the ISO file that you downloaded earlier.

2. Attach the ISO as a DVD drive:

   Add-VMDvdDrive -VMName "<VM_NAME>" -Path "<DRIVE>:\<PATH>.ISO"
   

Start the upgrade

1. Check the OS version before the upgrade:

   # Check the OS version BEFORE OS Upgrade:
   Get-ComputerInfo | Select-Object WindowsProductName, WindowsInstallationType, OSDisplayVersion, WindowsBuildLabEx | Format-Table -AutoSize
   

2. Initiate the upgrade. The upgrade process will take a while, and the node may reboot several times. For more information regarding command-line options, see Windows Setup Command-Line Options.

   $DVDDrive = "D:\" # update to the drive path the .ISO was mounted to
   $logDir = "C:\Temp\Upgrade-Logs"
   
   # Create a folder to store the upgrade logs:
   if(-not(Test-Path -Path $logDir -PathType Container)){
   $null = New-Item -Path $logDir -ItemType Directory
   }
   
   # /auto upgrade parameter to perform an in-place upgrade.
   # /dynamicupdate enables the download of updates during the upgrade process.
   # added " /quiet" and " /EULA accept" to arguments for the setup.exe command, to suppress the GUI and accept the EULA.
   
   Start-Process -FilePath "$DVDDrive\setup.exe" -ArgumentList "/auto upgrade /dynamicupdate enable /copylogs $logDir /quiet /eula accept"
   

3. Check the OS after the upgrade:

   # Check the OS version AFTER OS has been upgraded:
   Get-ComputerInfo | Select-Object WindowsProductName, WindowsInstallationType, OSDisplayVersion, WindowsBuildLabEx | Format-Table -AutoSize
   

Upgrade Hyper-V hosts

The upgrade process varies depending on the roles and services in your environment. If you have Storage Spaces Direct, clustering, or similar features, complete the necessary maintenance tasks to take a node offline for the upgrade process.

If virtual machines (VMs) aren't using clustering, evaluate live migrating VMs to another host within the environment depending on capacity.

After you put the Hyper-V host in the maintenance mode, follow the steps in Perform in-place upgrade. Repeat this process for all Hyper-V hosts within your cluster.

Upgrade Network Controller VMs (Service Fabric)

Before you upgrade or restart a Network Controller VM, disable it in the Service Fabric cluster.

Follow these steps on a remote computer that has WinRM connectivity to the Network Controller VMs.

1. Retrieve the current state of the Network Controller nodes:

   Get-SdnServiceFabricNode -NetworkController "<NC_VM>" | FT NodeName, IpAddressOrFQDN, NodeStatus, HealthState, IsStopped -AutoSize
   

2. Disable the node from the Service Fabric quorum. This step ensures Service Fabric can migrate primary replicas to other nodes and keep partition databases in sync.

   Disable-SdnServiceFabricNode -NetworkController "<NC_VM>" -NodeName "<Node_Name>"
   

3. After the node is safely disabled, perform the in-place upgrade Perform in-place upgrade.

4. After the node has completed the in-place upgrade, re-enable the node. This operation enables the node and waits until Service Fabric returns to a healthy state.

   Enable-SdnServiceFabricNode -NetworkController "<NC_VM>" -NodeName "<Node_Name>"
   

   If the operation times out, wait and manually check the state:

   Get-SdnServiceFabricNode -NetworkController "<NC_VM>" | FT NodeName, IpAddressOrFQDN, NodeStatus, HealthState, IsStopped -AutoSize
   
   Confirm-SdnServiceFabricHealthy -NetworkController "<NC_VM>"
   

   • If the commands return status healthy and node is up, repeat the process for other Network Controller nodes.

   • If the issue persists after an hour, see Troubleshooting.

5. Repeat the process for all Network Controller nodes in your deployment.

Perform Network Controller application update

After upgrading all Network Controller VMs, run the following command on one of the Network Controller VMs directly:

Update-NetworkController

This command initiates an application upgrade if it was not automatically initiated.

Upgrade Load Balancer Multiplexer VMs

You can upgrade Load Balancer Multiplexers without any additional requirements. To upgrade, proceed directly with Perform in-place upgrade on each Load Balancer Multiplexer, one at a time.

Upgrade gateway VMs

For instructions on how to upgrade gateway VMs, see Upgrade SDN gateway VMs.

Troubleshooting

This section lists common issues that you might encounter during the upgrade process and their recommended remediations.

Service Fabric node is not healthy

Issue

In some cases, the NetAdapter might get renamed during the upgrade. This causes issues as Network Controller node configuration requires the RestInterface to match the NetAdapter name on the VM.

Remediation

1. Verify that the NetAdapter on the Network Controller VM matches the configuration in NetworkController. For -NetworkController, specify a working Network Controller VM. For -Name, specify the non-working SDN node.

   Get-SdnNetworkControllerNode -NetworkController "<WORKING NC_VM>" -Name "<BROKEN NC_VM>"
   

2. Note the RestInterface value from the output.

3. Connect to Network Controller VM and run Get-NetAdapter directly. Ensure the name matches the value returned in step 1.

   • If the adapter name is changed, check if the previous adapter is orphaned or ghosted.

     Get-PnpDevice -Class net | Where-Object Status -eq Unknown | Select-Object FriendlyName,InstanceId
     

   • If an orphaned adapter exists, remove it.

     pnputil /remove-device "INSTANCE_ID"
     

   • Rename the new adapter back to the original name using Rename-NetAdapter.

Unable to resolve FQDN of Network Controllers

Issue

In some cases, unattend.xml is applied for initial VM deployment with SdnExpress. If DNS servers have changed since the initial deployment, incorrect DNS servers can be programmed into the adapters, causing FQDN resolution failures.

Remediation

1. Verify that you can resolve the FQDN of other Network Controller nodes. Ensure that the FQDN resolution succeeds because Service Fabric relies on FQDN for communication.

   Resolve-DnsName -Name "<NC NODE FQDN>" -Type A
   

2. If resolution fails, check the current DNS servers configured and confirm they match the configuration on other Network Controller VMs.

   Get-DnsClientServerAddress
   

3. If DNS servers differ, update them using Set-DnsClientServerAddress. If DNS servers are correct but resolution still fails, investigate your DNS infrastructure.

Resources reporting configurationState failures

Issue

During the upgrade process, you might encounter any of the following failures:

• Load Balancer Multiplexer VMs reporting configurationState failure.

• Servers reporting configurationState failure.

• Virtual Networks reporting configurationState failure.

Remediation

These errors are typically transient and can be resolved by moving the Service Fabric replicas for the affected service. Perform this operation directly on a Network Controller VM that is enabled within the Service Fabric cluster.

Move-SdnServiceFabricReplica -ServiceTypeName VSwitchService
Move-SdnServiceFabricReplica -ServiceTypeName GatewayManager
Move-SdnServiceFabricReplica -ServiceTypeName SlbManagerService

Traffic is unable to traverse the Gateway connection

Issue

Due to known issues in certain builds, performing an update or upgrade might result in stale route mappings for specific address prefixes on the Gateway connection.

Remediation

If you encounter data-path routing issues and resources don't report failures, we recommend rebooting the Gateway VM that hosts the Virtual Gateway or Network Connection.