Azure Stack HCI 23H2 Demo Lab - Azure connection: Not connected recently—Connected 3 days ago

Ahoi,

we're currently evaluating/building/exploring Azure Stack HCI and came across an issue after letting the cluster run for some time.

When checking our Single Node Cluster today I noticed that there is a connection error:

This is the alert

Unfortunately the troubleshooting information is not helpful for analysis of the issue.
This is the current status when executingGet-AzureStackHCI

This is the current version of the Node when executing Get-StampInformation

When executing Sync-AzureStackHCI I can see the following log entries under \\ATVIEHCIN001\c$\ProgramData\AzureConnectedMachineAgent\Log\himds.log

time="2024-09-27T17:01:25+02:00" level=debug msg="Received REST request" apiver=2019-03-11 host=localhost method=GET path=/metadata/instance uuid=faa4ce35-7fa2-4cc8-bffe-e3826daa8246 time="2024-09-27T17:01:25+02:00" level=debug msg="Tinyrest Utility Logger" available=9.00 limit=40 limiter=metadatalimiter time="2024-09-27T17:01:25+02:00" level=info msg="Successfully sent the 'metadata'" sessionId=faa4ce35-7fa2-4cc8-bffe-e3826daa8246 time="2024-09-27T17:01:25+02:00" level=debug msg="Received REST request" apiver=2019-03-11 host=localhost method=GET path=/metadata/instance uuid=f83d7ac6-fe09-42ea-95be-4a75834f7f57 time="2024-09-27T17:01:25+02:00" level=debug msg="Tinyrest Utility Logger" available=8.46 limit=40 limiter=metadatalimiter time="2024-09-27T17:01:25+02:00" level=info msg="Successfully sent the 'metadata'" sessionId=f83d7ac6-fe09-42ea-95be-4a75834f7f57 time="2024-09-27T17:01:25+02:00" level=debug msg="Received REST request" apiver=2019-03-11 host=localhost method=GET path=/metadata/instance uuid=6544386f-be5b-4b87-935d-4be19a0d3e16 time="2024-09-27T17:01:25+02:00" level=debug msg="Tinyrest Utility Logger" available=7.82 limit=40 limiter=metadatalimiter time="2024-09-27T17:01:25+02:00" level=info msg="Successfully sent the 'metadata'" sessionId=6544386f-be5b-4b87-935d-4be19a0d3e16 time="2024-09-27T17:01:25+02:00" level=debug msg="Received REST request" apiver=2019-03-11 host=localhost method=GET path=/metadata/instance uuid=38152c2d-7763-4946-8ed6-5ed92c67719a time="2024-09-27T17:01:25+02:00" level=debug msg="Tinyrest Utility Logger" available=7.18 limit=40 limiter=metadatalimiter time="2024-09-27T17:01:25+02:00" level=info msg="Successfully sent the 'metadata'" sessionId=38152c2d-7763-4946-8ed6-5ed92c67719a time="2024-09-27T17:01:25+02:00" level=debug msg="Received REST request" apiver=2019-08-15 host=localhost method=GET path=/metadata/identity/oauth2/token uuid=9c768b3d-dbe6-469d-8e02-62929ab9fa26 time="2024-09-27T17:01:25+02:00" level=debug msg="Tinyrest Utility Logger" available=9.00 limit=40 limiter=tokenlimiter time="2024-09-27T17:01:25+02:00" level=debug msg="Validating url parameters" api-version=2019-08-15 resource="https://management.core.windows.net/" sessionId=9c768b3d-dbe6-469d-8e02-62929ab9fa26 type="access token" time="2024-09-27T17:01:25+02:00" level=warning msg="Invalid basic authentication header" session=9c768b3d-dbe6-469d-8e02-62929ab9fa26 time="2024-09-27T17:01:25+02:00" level=info msg="Missing Basic Authorization header" reason=unauthorized_client sessionId=9c768b3d-dbe6-469d-8e02-62929ab9fa26 time="2024-09-27T17:01:25+02:00" level=debug msg="Received REST request" apiver=2019-08-15 host=localhost method=GET path=/metadata/identity/oauth2/token uuid=0eba8778-c346-4927-bd7c-76a0d856363b time="2024-09-27T17:01:25+02:00" level=debug msg="Tinyrest Utility Logger" available=9.00 limit=40 limiter=tokenlimiter time="2024-09-27T17:01:25+02:00" level=debug msg="Validating url parameters" api-version=2019-08-15 resource="https://management.core.windows.net/" sessionId=0eba8778-c346-4927-bd7c-76a0d856363b type="access token" time="2024-09-27T17:01:25+02:00" level=debug msg="Responding with a token" sessionId=0eba8778-c346-4927-bd7c-76a0d856363b time="2024-09-27T17:01:25+02:00" level=info msg="Acquire access token for https://management.core.windows.net/" session=0eba8778-c346-4927-bd7c-76a0d856363b time="2024-09-27T17:01:25+02:00" level=debug msg="searching the token in cache for https://management.core.windows.net/" session=0eba8778-c346-4927-bd7c-76a0d856363b time="2024-09-27T17:01:25+02:00" level=debug msg="Found the token in the cache 'eyJ0e' for the audience 'https://management.core.windows.net/'." session=0eba8778-c346-4927-bd7c-76a0d856363b time="2024-09-27T17:01:25+02:00" level=debug msg="Preparing to send a token" sessionId=0eba8778-c346-4927-bd7c-76a0d856363b time="2024-09-27T17:01:25+02:00" level=info msg="Successfully sent the token, which expires at: 2024-09-28 16:55:47 +0200 CEST, or in 86061 seconds" sessionId=0eba8778-c346-4927-bd7c-76a0d856363b time="2024-09-27T17:01:25+02:00" level=debug msg="Received REST request" apiver=2019-03-11 host=localhost method=GET path=/metadata/instance uuid=91381749-836b-453d-8a98-72d4f283d107 time="2024-09-27T17:01:25+02:00" level=debug msg="Tinyrest Utility Logger" available=8.15 limit=40 limiter=metadatalimiter time="2024-09-27T17:01:25+02:00" level=info msg="Successfully sent the 'metadata'" sessionId=91381749-836b-453d-8a98-72d4f283d107 time="2024-09-27T17:01:38+02:00" level=debug msg="Received REST request" apiver=2019-03-11 host=localhost method=GET path=/metadata/instance uuid=be897522-6bd7-4d80-9f99-f8488bd5c3f6 time="2024-09-27T17:01:38+02:00" level=debug msg="Tinyrest Utility Logger" available=9.00 limit=40 limiter=metadatalimiter time="2024-09-27T17:01:38+02:00" level=info msg="Successfully sent the 'metadata'" sessionId=be897522-6bd7-4d80-9f99-f8488bd5c3f6 time="2024-09-27T17:01:38+02:00" level=debug msg="Received REST request" apiver=2019-03-11 host=localhost method=GET path=/metadata/instance uuid=484eb470-4a83-467f-a3fb-6dad6a6284a1 time="2024-09-27T17:01:38+02:00" level=debug msg="Tinyrest Utility Logger" available=8.33 limit=40 limiter=metadatalimiter time="2024-09-27T17:01:38+02:00" level=info msg="Successfully sent the 'metadata'" sessionId=484eb470-4a83-467f-a3fb-6dad6a6284a1 time="2024-09-27T17:01:38+02:00" level=debug msg="Received REST request" apiver=2019-03-11 host=localhost method=GET path=/metadata/instance uuid=bc43ed7d-15a8-456e-bcb3-74c3de016e2c time="2024-09-27T17:01:38+02:00" level=debug msg="Tinyrest Utility Logger" available=7.70 limit=40 limiter=metadatalimiter time="2024-09-27T17:01:38+02:00" level=info msg="Successfully sent the 'metadata'" sessionId=bc43ed7d-15a8-456e-bcb3-74c3de016e2c time="2024-09-27T17:01:38+02:00" level=debug msg="Received REST request" apiver=2019-03-11 host=localhost method=GET path=/metadata/instance uuid=6e997dc8-c145-4a53-b27e-ebdc0b5ceddb time="2024-09-27T17:01:38+02:00" level=debug msg="Tinyrest Utility Logger" available=7.04 limit=40 limiter=metadatalimiter time="2024-09-27T17:01:38+02:00" level=info msg="Successfully sent the 'metadata'" sessionId=6e997dc8-c145-4a53-b27e-ebdc0b5ceddb time="2024-09-27T17:01:38+02:00" level=debug msg="Received REST request" apiver=2019-08-15 host=localhost method=GET path=/metadata/identity/oauth2/token uuid=796b8851-23d6-4306-98f1-2c38145eba93 time="2024-09-27T17:01:38+02:00" level=debug msg="Tinyrest Utility Logger" available=9.00 limit=40 limiter=tokenlimiter time="2024-09-27T17:01:38+02:00" level=debug msg="Validating url parameters" api-version=2019-08-15 resource="https://management.core.windows.net/" sessionId=796b8851-23d6-4306-98f1-2c38145eba93 type="access token" time="2024-09-27T17:01:38+02:00" level=warning msg="Invalid basic authentication header" session=796b8851-23d6-4306-98f1-2c38145eba93 time="2024-09-27T17:01:38+02:00" level=info msg="Missing Basic Authorization header" reason=unauthorized_client sessionId=796b8851-23d6-4306-98f1-2c38145eba93 time="2024-09-27T17:01:38+02:00" level=debug msg="Received REST request" apiver=2019-08-15 host=localhost method=GET path=/metadata/identity/oauth2/token uuid=e7ac8a39-87d6-4fab-996f-3754aabbe58e time="2024-09-27T17:01:38+02:00" level=debug msg="Tinyrest Utility Logger" available=9.00 limit=40 limiter=tokenlimiter time="2024-09-27T17:01:38+02:00" level=debug msg="Validating url parameters" api-version=2019-08-15 resource="https://management.core.windows.net/" sessionId=e7ac8a39-87d6-4fab-996f-3754aabbe58e type="access token" time="2024-09-27T17:01:38+02:00" level=debug msg="Responding with a token" sessionId=e7ac8a39-87d6-4fab-996f-3754aabbe58e time="2024-09-27T17:01:38+02:00" level=info msg="Acquire access token for https://management.core.windows.net/" session=e7ac8a39-87d6-4fab-996f-3754aabbe58e time="2024-09-27T17:01:38+02:00" level=debug msg="searching the token in cache for https://management.core.windows.net/" session=e7ac8a39-87d6-4fab-996f-3754aabbe58e time="2024-09-27T17:01:38+02:00" level=debug msg="Found the token in the cache 'eyJ0e' for the audience 'https://management.core.windows.net/'." session=e7ac8a39-87d6-4fab-996f-3754aabbe58e time="2024-09-27T17:01:38+02:00" level=debug msg="Preparing to send a token" sessionId=e7ac8a39-87d6-4fab-996f-3754aabbe58e time="2024-09-27T17:01:38+02:00" level=info msg="Successfully sent the token, which expires at: 2024-09-28 16:55:47 +0200 CEST, or in 86048 seconds" sessionId=e7ac8a39-87d6-4fab-996f-3754aabbe58e time="2024-09-27T17:01:38+02:00" level=debug msg="Received REST request" apiver=2019-03-11 host=localhost method=GET path=/metadata/instance uuid=9cdf311b-9ac4-4c88-bf2e-368c1dbff065 time="2024-09-27T17:01:38+02:00" level=debug msg="Tinyrest Utility Logger" available=7.81 limit=40 limiter=metadatalimiter time="2024-09-27T17:01:38+02:00" level=info msg="Successfully sent the 'metadata'" sessionId=9cdf311b-9ac4-4c88-bf2e-368c1dbff065

Do you have experienced this as well or have any idea where to look and troubleshoot?

BR Maximilian