Usare lo strumento di controllo diagnostico per diagnosticare e risolvere i problemi di ambiente per gli errori di creazione del cluster AKS (anteprima)

Può essere difficile identificare i problemi correlati all'ambiente, ad esempio la configurazione di rete, che possono causare un errore di creazione del cluster del servizio Azure Kubernetes. Il controllo diagnostico è uno strumento di PowerShell che può aiutare a identificare i fallimenti nella creazione di cluster AKS a causa di potenziali problemi nell’ambiente.

Operazioni preliminari

Prima di iniziare, assicurarsi di disporre dei prerequisiti seguenti. Se non si soddisfano i requisiti per l'esecuzione dello strumento di verifica diagnostica, inviare una richiesta di supporto:

• Accesso diretto al cluster locale di Azure in cui è stato creato il cluster del servizio Azure Kubernetes. Questo accesso può essere tramite Desktop remoto (RDP) oppure è anche possibile accedere a uno dei nodi fisici locali di Azure.
• Esaminare i concetti di rete per la creazione di un cluster del servizio Azure Kubernetes e l'architettura del cluster del servizio Azure Kubernetes.
• Nome della rete logica collegata al cluster del servizio Azure Kubernetes.
• Una chiave privata SSH per il cluster del servizio Azure Kubernetes, usata per accedere alla macchina virtuale del nodo del piano di controllo del cluster del servizio Azure Kubernetes.

Ottenere l'indirizzo IP della macchina virtuale del nodo del piano di controllo del cluster del servizio Azure Kubernetes

Eseguire il comando seguente da un nodo fisico nel cluster locale di Azure. Assicurarsi di passare il nome e non l'ID di Azure Resource Manager del cluster del servizio Azure Kubernetes:

invoke-command -computername (get-clusternode) -script {get-vmnetworkadapter -vmname *} | Where-Object {$_.Name -like "$cluster_name*control-plane-*"} | select vmname, ipaddresses

Output previsto:

VMName                                                 IPAddresses
------                                                 -----------
<cluster-name>-XXXXXX-control-plane-XXXXXX {172.16.0.10, 172.16.0.4, fe80::ec:d3ff:fea0:1}

Se non viene visualizzata una macchina virtuale del piano di controllo come illustrato nell'output precedente, inviare una richiesta di supporto.

Se viene visualizzata una macchina virtuale del piano di controllo con:

• 0 Indirizzi IPv4: inviare una richiesta di supporto.
• 1 Indirizzo IP: usare l'indirizzo IPv4 come input per vmIP il parametro.
• 2 indirizzi IP: usare uno qualsiasi degli indirizzi IPv4 come input per vmIP il parametro nel controllo di diagnostica.

Eseguire lo script del controllo di diagnostica

Copiare lo script di PowerShell seguente denominato run_diagnostic.ps1 in un nodo del cluster locale di Azure:

<#
.SYNOPSIS
    Runs diagnostic checker tool in target cluster control plane VM and returns the result.

    This script runs the following tests from target cluster control plane VM:
    1. cloud-agent-connectivity-test: Checks whether the DNS server can resolve the Moc cloud agent FQDN and that the cloud agent is reachable from the control plane node VM. Cloud agent is created using one of the IP addresses from the [management IP pool](hci/plan/cloud-deployment-network-considerations.md#management-ip-pool), on port 55000. The control plane node VM is given an IP address from the Arc VM logical network.
    2. gateway-icmp-ping-test: Checks whether the gateway specified in the logical network attached to the AKS cluster is reachable from the AKS cluster control plane node VM.
    3. http-connectivity-required-url-test: Checks whether the required URLs are reachable from the AKS cluster control plane node VM.

.DESCRIPTION
    This script transfers a file from the local machine to a remote server using the SCP (Secure Copy Protocol) command.

.PARAMETER lnetName
    The name of the LNET used for the cluster.

.PARAMETER sshPath
    The path to the private SSH key for the target cluster.

.PARAMETER vmIP
    IP of the target cluster control plane VM.


.EXAMPLE
    .\run_diagnostic.ps1 -lnetName lnet1 -sshPath C:\Users\test\.ssh\test-ssh.pem -vmIP "172.16.0.10"
    This example runs diagnostic checker tool in the VM with IP 172.16.0.10 using ssh key C:\Users\test\.ssh\test-ssh.pem and outputs the result.
#>

param (
    [Parameter(Mandatory=$true)]
    [string]$lnetName,

    [Parameter(Mandatory=$true)]
    [string]$sshPath,

    [Parameter(Mandatory=$true)]
    [string]$vmIP
)

$urlArray = @(
    "https://management.azure.com",
    "https://eastus.dp.kubernetesconfiguration.azure.com",
    "https://login.microsoftonline.com",
    "https://eastus.login.microsoft.com",
    "https://login.windows.net",
    "https://mcr.microsoft.com",
    "https://gbl.his.arc.azure.com",
    "https://k8connecthelm.download.prss.microsoft.com",
    "https://guestnotificationservice.azure.com",
    "https://sts.windows.net",
    "https://graph.microsoft.com"
)
$urlList=$urlArray -join ","

# check vm is reachable
try {
  $pingResult = Test-Connection -ComputerName $vmIP -Count 1 -ErrorAction Stop
  if ($pingResult.StatusCode -eq 0) {
    Write-Host "Connection to $vmIP succeeded."
  } else {
    Write-Host "Connection to AKS cluster control plane VM $vmIP failed with status code: $($pingResult.StatusCode). Please make sure AKS cluster control plane VM $vmIP is reachable from the host"
    exit
  }
} catch {
  Write-Host "Connection to AKS cluster control plane VM $vmIP failed. Please make sure AKS cluster control plane VM $vmIP is reachable from the host"
  Write-Host "Exception message: $_"
  exit
}

# retreiving LNET 
$lnet=get-mocvirtualnetwork -group Default_Group -name $lnetName
# getting gateway address from LNET
$gateway=$lnet.properties.subnets[0].properties.routeTable.properties.routes[0].properties.nextHopIpAddress
if (-not $gateway) {
  Write-Error "Check Gateway address in the AKS logical network $lnetName"
  exit
}

# getting cloudfqdn from archciconfig
$arcHCIConfig=get-archciconfig
$cloudFqdn="http://"+$arcHCIConfig.Item('cloudFQDN')+":55000" 

$configContent = @"
checks:
- metadata:
    creationTimestamp: null
    name: cloud-agent-connectivity-test
  parameters:
    hostnames: <CLOUD_FQDN>
    skipeof: "true"
  type: HTTPConnectivity
- metadata:
    annotations:
      skip-error-on-failure: "true"
    creationTimestamp: null
    name: gateway-icmp-ping-test
  parameters:
    ips: <GATEWAY>
    packetLossThreshold: "20"
  type: ICMPPing
- metadata:
    creationTimestamp: null
    name: http-connectivity-required-url-test
  parameters:
    hostnames: <URL_LIST>
  type: HTTPConnectivity
exports:
- metadata:
    creationTimestamp: null
  parameters:
    filelocation: /home/clouduser/results.yaml
  type: FileSystem
metadata:
  creationTimestamp: null
"@

# update config file with the values of cloud fqdn, gateway and dns servers
$configContent = $configContent.replace("<CLOUD_FQDN>", $cloudFqdn)
$configContent = $configContent.replace("<GATEWAY>", $gateway)
$configContent = $configContent.replace("<URL_LIST>", $urlList)

$filePath = "config.yaml"
# Write to config.yaml
Set-Content -Path $filePath -Value $configContent

$dest = 'clouduser@' + $vmIP + ":config.yaml"

# Copy the config file to target cluster VM
Write-Host "Copying test config file to target cluster VM...."
$command = "scp -i $sshPath -o StrictHostKeyChecking=no -o BatchMode=yes config.yaml $dest"
try {
  $output=invoke-expression $command
  if ($LASTEXITCODE -ne 0) {
    Write-Error "Couldn't ssh to AKS cluster control plane VM $vmIP. Please check the ssh key"
    exit
  }
} catch {
  Write-Host "Couldn't ssh to AKS cluster control plane VM $vmIP. Please check the ssh key"
  Write-Host "Exception message: $_"
  exit
}
Write-Output "Copied config.yaml successfully."
$runScriptContent = @"
sudo su - root -c "/usr/bin/diagnostics-checker -c /home/clouduser/config.yaml"
"@

$filePath = "run_diag.sh"

Set-Content -Path $filePath -Value $runScriptContent
$dest = 'clouduser@' + $vmIP + ":run_diag.sh"
scp -i $sshPath -o StrictHostKeyChecking=no -o BatchMode=yes run_diag.sh $dest

$dest = 'clouduser@' + $vmIP
ssh -i $sshPath $dest -o StrictHostKeyChecking=no -o BatchMode=yes 'chmod +x run_diag.sh'

$sedCommand="sed -i -e 's/\r$//' run_diag.sh"
ssh -i $sshPath -o StrictHostKeyChecking=no -o BatchMode=yes $dest $sedCommand

if (Test-Path -Path "results.yaml") {
  Remove-Item results.yaml
}

ssh -i $sshPath -o StrictHostKeyChecking=no -o BatchMode=yes $dest './run_diag.sh'
ssh -i $sshPath -o StrictHostKeyChecking=no -o BatchMode=yes $dest "sudo su - root -c 'chmod a+r /home/clouduser/results.yaml'"

$src= 'clouduser@' + $vmIP + ":results.yaml"
scp -i $sshPath -o StrictHostKeyChecking=no -o BatchMode=yes $src results.yaml

if (-Not (Test-Path -Path "results.yaml")) {
  write-host "Test failed to perform"
  exit
}

Install-Module powershell-yaml

$resultContent = Get-Content -path results.yaml | ConvertFrom-Yaml

$testResults = @()
$cloudAgentRecommendation = @"
Make sure that the logical network IP addresses can connect to all the management IP pool addresses on the required ports. Check AKS network port and cross vlan requirements for detailed list of ports that need to be opened.
"@
$gatewayRecommendation = @"
- Ensure gateway is operational
- Verify routing configurations
- Adjust firewall rules to allow ICMP traffic
"@
$urlRecommendation = @"
Ensure that the logical network IP addresses have outbound internet access. If there's a firewall, ensure that AKS required URLs are accessible from Arc VM logical network.
"@


foreach ($check in $resultContent.spec.checks) {
  if ($check.result.outcome -like "Success") {
    $recommendation=""
  }elseif ($check.metadata.name -like "cloud-agent-connectivity-test") {
    $recommendation=$cloudAgentRecommendation
  }elseif ($check.metadata.name -like "gateway-icmp-ping-test") {
    $recommendation=$gatewayRecommendation
  }elseif ($check.metadata.name -like "http-connectivity-required-url-test") {
    $recommendation=$urlRecommendation
  }
  $testResults += [PSCustomObject]@{
    TestName=$check.metadata.name
    Outcome= $check.result.outcome
    Recommendation = $recommendation
  }
}

$testResults | Format-Table -Wrap -AutoSize

Output di esempio:

TestName                            Outcome Recommendation
--------                            ------- --------------
cloud-agent-connectivity-test       Success
gateway-icmp-ping-test              Success 
http-connectivity-required-url-test Failure Ensure that the logical network IP addresses have outbound internet access. If there's a firewall, ensure that AKS required URLs are accessible from Arc VM logical network.

Analizzare l'output del controllo di diagnostica

Nella tabella seguente viene fornito un riepilogo di ogni test eseguito dallo script, incluse le possibili cause di errori e raccomandazioni per la mitigazione:

Passaggi successivi

Se il problema persiste, raccogli i log del cluster AKS prima di creare una richiesta di supporto.