Error in creating Failover Clutser - Windows Server 2012 R2

Question

_{Monday, November 17, 2014 5:21 AM}

Hello,

I have created a single node Failover Cluster in Windows Server 2012 R2. When I am trying to add second node to this cluster, it gives the error: “Cluster service on node Node1did not reach the running state. The error code is 0x5b4. For more information check the cluster log and the system event log from node Node1. This operation returned because the timeout period expired.

The server 'Node1.mydomain.com' could not be added to the cluster.
An error occurred while adding node 'Node1.mydomain.com' to cluster 'My_Cluster'.

This operation returned because the timeout period expired.”

I have attached the screenshots of Event ID.

Firewalls are turned off on both servers (Node1 and Node2). I have successfully created single node clusters on both servers, but when I try to add second node, it gives error.

Before starting the creation of new cluster I don't forget to Destroy previous cluster and run "Clear-ClusterNode".

Every time they successfully pass the Validate Configuration Test, but give the error on creating failover cluster.

I'm using the service account which has full permissions on CNU and both nodes, plus it has permissions to create computer accounts in AD.

Can anyone please help?

Thank you.

Best regards,

Hasan Bin Hasib

All replies (13)

_{Monday, November 17, 2014 7:29 AM}

Verify that your cluster is reachable through the cluster name/address (try to ping My_Cluster from the new node)

Gleb.

_{Tuesday, November 18, 2014 4:13 AM}

Yes, I tried to ping the CNU 'My_Cluster' from every node/host and other servers, and the ping response was fine (i.e. My_Cluster was reachable)  from every machine on domain.

One important point is that the these two nodes lie on two different hosts. There are no Firewall policies/Antivirus neither on hosts nor on nodes.

~ Hasan Bin Hasib

_{Tuesday, November 18, 2014 7:05 AM}

Did you try disabling cluster's Virtual Adapter Performance Filter as described in this kb article?

Gleb.

_{Tuesday, November 18, 2014 6:41 PM}

What are the IP configurations on the two different nodes you are trying to form into a cluster?

. : | : . : | : . tim

_{Wednesday, November 19, 2014 4:33 AM}

Hi Hasan Bin Hasib,

Are you using the same NIC for management and storage? I have experienced the same issue before please use the dedicated nic for management or storage.

The related KB:

Network Recommendations for a Hyper-V Cluster in Windows Server 2012

http://technet.microsoft.com/en-us/library/dn550728.aspx

I’m glad to be of help to you!

We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time.
Thanks for helping make community forums a great place.

_{Wednesday, November 19, 2014 7:54 AM}

Thank you fellows!

The issue just resolved when I moved the Node1 and Node2 on the same host, and the cluster was successfully created.

After the creation of cluster, as soon as I moved the Node2 to some other host, then the Failover Cluster Manager console started showing Node2 is Down. Error: Cluster node Node2 could not to join the cluster because it failed to communicate over the network with any other node in the cluster. Verify the network connectivity and configuration of any network firewalls. Event ID: 1653

IP Address of Node1: 172.16.1.186 ; IP Address of Node2: 172.16.1.187

And yes, the ‘Microsoft Failover Cluster Virtual Adapter Performance Filter’ is already disabled on all virtual adapters. All firewalls are disabled, no antivirus installed. I’m using same type of NICs.

I don’t know where the blockage is, and I am badly stuck. Please help.

Thank you.

~ Hasan Bin Hasib

_{Wednesday, November 19, 2014 10:34 AM}

Can anyone please tell me what are the port requirements?

I mean all of those Port numbers required to be be open for the cluster communication of Hyper-V VMs hosted on different physical machines (in Windows Server 2012 R2).

Thank you

~ Hasan Bin Hasib

_{Wednesday, November 19, 2014 4:04 PM}

"After the creation of cluster, as soon as I moved the Node2 to some other host, then the Failover Cluster Manager console started showing Node2 is Down. Error: Cluster node Node2 could not to join the cluster because it failed to communicate over the network with any other node in the cluster. Verify the network connectivity and configuration of any network firewalls. Event ID: 1653"

Sounds like you have a networking issue.  Default firewall configuration on the hosts should allow communication, so unless you have created some specific restrictions, your firewall should not be an issue.  Easy to test - disable firewalls.

But the first thing to try is to put the two VMs on different hosts and simply try to ping each other.  If you can't ping, you need to figure out why the VMs can't see each other.  Note that by default, ping is disabled via firewall rules.  Ping will work to a domain, but it will not work between machines without enabling the ICMP echo inbound/outbound rules in the Windows Firewall.  Enabling these rules before creating a cluster is not a requirement, but if you simply want to ping between machines, you have to enable them.

. : | : . : | : . tim

_{Wednesday, November 19, 2014 4:34 PM}

You mentioned that local firewall is turned off on the nodes, so, since nodes are on the same subnet,  this has nothing to do with the opening of ports through the firewall. Double check local fw state, make sure it's turned off for all profiles. If the nodes are connected to multiple networks,  make sure these networks are available on both hosts. The cause may be as trivial as mistyping vlan number when configuring trunk on uplink ports...

Gleb.

_{Thursday, November 20, 2014 4:00 AM}

Dear Tim,

The two VMs lie on the different hosts. Ping response is always coming from Node1 to Node2 and vice versa.

Local firewalls are turned off for all profiles on both VMs and both hosts.

~ Hasan Bin Hasib

_{Thursday, November 20, 2014 4:03 AM}

Dear Gleb,

I have double checked and strictly ensured that local firewalls are turned off for all profiles on both VMs and both hosts.

The nodes are connected through single network.

~ Hasan Bin Hasib

_{Thursday, November 20, 2014 6:53 AM}

Then you have to continue inverstigating connectivity issues:

Check ping between nodes with packet size larger than MTU ( -l 2000 will do); use netstat or Resource Monitor to check which connections hang when you recieve the error message.

Gleb.

_{Friday, November 21, 2014 9:15 AM}

I have checked the Listening Ports from Resource Monitor.

The Firewall Status of Cluster Service is "Allowed, restricted".

What does it mean? Can you please help.

I have attached the snap.

~ Hasan Bin Haseeb